Metadata-Version: 2.4
Name: OrderPulse
Version: 0.2.49
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Rust
Classifier: License :: Other/Proprietary License
Classifier: Operating System :: POSIX :: Linux
Summary: High-performance exchange feed parser and orderflow analytics engine with Rust and Python bindings
Requires-Python: >=3.9
Description-Content-Type: text/markdown

# fastreader / OrderPulse

High-performance Python bindings for reading NSE order and trade binary feed files, enriching messages with contract metadata, and building order book snapshots.

`fastreader` is designed for Python users who need Rust-speed parsing with a simple Jupyter-friendly API. The Rust backend reads NSE binary feed files, decodes order/trade packets, exposes each message as a Python dictionary through PyO3, optionally enriches tokens using a symbol master CSV, and builds market-depth/order-book outputs.

> Note: All example outputs in this README use realistic placeholder values. Actual values depend on your binary feed file and contract master CSV.

---

## 1. What this library does

Use `fastreader` / `OrderPulse` when you want to:

- Parse NSE binary order and trade feed files from Python.
- Load all messages into memory for fast repeated analysis.
- Stream large feed files one message at a time without loading everything into memory.
- Convert Rust-decoded feed packets into Python dictionaries.
- Filter order and trade messages.
- Enrich messages with token metadata such as symbol, strike, option type, expiry, lot size, and contract name.
- Build an order book from cached messages or from a streaming reader.
- Export top-of-book and market-depth snapshots.
- Construct standard feed file paths for NSE CM and FO streams.

---

## 2. Architecture overview

The library follows this flow:

```text
Binary feed file
    -> Rust binary parser
    -> Python message dictionaries
    -> optional SymbolMaster enrichment
    -> OrderbookBuilder
    -> snapshot / full-depth output
```

Main internal components:

| Component | Purpose |
|---|---|
| Rust parser | Reads binary files and decodes order/trade packets. |
| PyO3 bindings | Exposes Rust classes and methods to Python. |
| `MessageCacheReader` | Loads all decoded messages into memory for repeated analysis. |
| `StreamingBinaryLoader` | Opens a file and reads messages sequentially using `get_next_msg()`. |
| `SymbolMaster` | Loads NSE contract master CSV and maps token to contract metadata. |
| `OrderbookBuilder` | Processes order/trade messages and builds token-wise depth. |
| `FeedPathBuilder` | Creates standard NSE feed file paths from segment/date/stream inputs. |

---

## 3. Public API

The Python module exports these classes:

```python
from fastreader import (
    MessageCacheReader,
    StreamingBinaryLoader,
    OrderbookBuilder,
    SymbolMaster,
    FeedPathBuilder,
)
```

---

## 4. Installation

### From PyPI

```bash
pip install fastreader
```

### From a local source checkout

```bash
pip install maturin
maturin develop --release
```

For a production wheel build:

```bash
maturin build --release
pip install target/wheels/*.whl
```

### Verify installation

```python
import fastreader
from fastreader import MessageCacheReader, StreamingBinaryLoader, OrderbookBuilder, SymbolMaster, FeedPathBuilder

print("fastreader imported successfully")
```

Expected output:

```text
fastreader imported successfully
```

---

## 5. Required input files

### 5.1 Binary feed file

The binary file must contain supported NSE feed messages. The parser recognizes these message types:

| Message type | Meaning in this library |
|---|---|
| `N` | New order message |
| `M` | Modify order message |
| `X` | Cancel/delete order message |
| `T` | Trade message |

The binary header must be valid. If the file is empty, truncated, or starts with an unsupported first message type, the loader raises a runtime error.

### 5.2 Symbol master CSV file

`SymbolMaster` expects a CSV with these columns:

| Required column | Used as |
|---|---|
| `FinInstrmId` | Token / instrument id |
| `TckrSymb` | Symbol, for example `NIFTY` |
| `XpryDt` | Expiry timestamp, converted to readable date |
| `StrkPric` | Strike price; divided by 100 |
| `OptnTp` | Option type, for example `CE`, `PE`, or `XX` |
| `StockNm` | Full contract name |

Optional lot-size column:

- `NewBrdLotQty`, preferred
- `MinLot`, fallback

---

## 6. Message dictionary format

Order messages returned by `get_order_message()` or `get_next_msg()` look like this:

```python
{
    "message_kind": "order",
    "seq_no": 42,
    "msg_len": 48,
    "stream_id": 2,
    "msg_type": "N",
    "exch_ts": 1716269400123456789,
    "local_ts": 1716269401123456789,
    "order_id": 12345678901,
    "token": 40434,
    "order_type": "B",
    "price": 2195050,
    "quantity": 50,
    "flags": False,
    "token_symbol": None,
    "strike_price": None,
    "option_type": None,
}
```

Trade messages look like this:

```python
{
    "message_kind": "trade",
    "seq_no": 99,
    "msg_len": 48,
    "stream_id": 2,
    "msg_type": "T",
    "exch_ts": 1716269402123456789,
    "local_ts": 1716269403123456789,
    "buy_order_id": 111111111,
    "sell_order_id": 222222222,
    "token": 40434,
    "trade_price": 2195100,
    "trade_quantity": 50,
    "flags": False,
    "token_symbol": None,
    "strike_price": None,
    "option_type": None,
}
```

After symbol enrichment, additional keys can be available:

```python
{
    "token_symbol": "NIFTY",
    "strike_price": 21950,
    "option_type": "CE",
    "expiry": "30-May-2024",
    "lot_size": 50,
    "name": "NIFTY24MAY21950CE",
}
```

---

## 7. Typical workflow

```python
from fastreader import MessageCacheReader, SymbolMaster, OrderbookBuilder

feed_path = "/data/NSE_FO/Feed_FO_StreamID_1_21_05_2026.bin"
symbol_csv = "/data/CONTRACT/21_05_2026/NSE_FO_contract_21052026.csv"

def show(value):
    print(value)

# 1. Load symbol metadata
sm = SymbolMaster()
sm_count = sm.load(symbol_csv)
show(sm_count)

# 2. Load binary messages into cache
reader = MessageCacheReader()
message_count = reader.load_to_cache(feed_path)
show(reader.get_cache_summary())

# 3. Build order book
builder = OrderbookBuilder()
processed = builder.build_from_list(reader)
show(processed)

# 4. Inspect active tokens and snapshot
active_tokens = builder.get_active_tokens()
show(active_tokens[:5])

if active_tokens:
    token = active_tokens[0]
    show(builder.get_snapshot(token, levels=5))
```

Expected output:

```text
95632
{'file_source': '/data/NSE_FO/Feed_FO_StreamID_1_21_05_2026.bin', 'total_messages': 1250000, 'total_orders': 1180000, 'total_trades': 70000, 'memory_usage_bytes': 80000000}
1180000
[40434, 40435, 40436, 50120, 50121]
{'token': 40434, 'found': True, 'mid_price': 2195075, 'best_bid': (2195050, 100), 'best_ask': (2195100, 50), 'spread': 50, 'bids': [(2195050, 100), ...], 'asks': [(2195100, 50), ...]}
```

---

# 8. `MessageCacheReader`

`MessageCacheReader` loads the entire binary feed into memory. Use it when the file is small enough to fit comfortably in RAM or when you need to query the same file multiple times.

## 8.1 Create a reader

```python
from fastreader import MessageCacheReader

reader = MessageCacheReader()
print(reader.get_cache_summary())
```

Expected output:

```text
{'file_source': None, 'total_messages': 0, 'total_orders': 0, 'total_trades': 0, 'memory_usage_bytes': 0}
```

## 8.2 `load_to_cache(file_path)`

Loads a binary feed file into memory.

Parameters:

| Parameter | Type | Required | Description |
|---|---:|---:|---|
| `file_path` | `str` | Yes | Path to the NSE binary feed file. |

Returns:

- `int`: number of decoded messages loaded.

Possible errors:

- File does not exist.
- File is empty.
- Invalid first message type.
- Truncated header or payload.

Example:

```python
from fastreader import MessageCacheReader

feed_path = "/data/NSE_FO/Feed_FO_StreamID_1_21_05_2026.bin"

reader = MessageCacheReader()
count = reader.load_to_cache(feed_path)

print("Loaded messages:", count)
```

Expected output:

```text
Loaded messages: 1250000
```

## 8.3 `get_all_messages()`

Returns every cached message as a human-readable string. Use this for quick inspection, logging, or debugging.

Parameters: none.

Returns:

- `list[str]`

Example:

```python
messages = reader.get_all_messages()

print("Total formatted messages:", len(messages))
print(messages[0])
```

Expected output:

```text
Total formatted messages: 1250000
Order Message: SeqNo 42, MsgLen 48, MsgType 'N', ExchTs 1716269400123456789, LocalTs 1716269401123456789, OrderId 12345678901, Token 40434, Side 'B', Price 2195050, Quantity 50, Missed 0
```

## 8.4 `get_order_message()`

Returns only order messages as Python dictionaries.

Parameters: none.

Returns:

- `list[dict]`

Example:

```python
orders = reader.get_order_message()

print("Order count:", len(orders))
print(orders[0])
```

Expected output:

```text
Order count: 1180000
{'message_kind': 'order', 'seq_no': 42, 'msg_len': 48, 'stream_id': 2, 'msg_type': 'N', 'exch_ts': 1716269400123456789, 'local_ts': 1716269401123456789, 'order_id': 12345678901, 'token': 40434, 'order_type': 'B', 'price': 2195050, 'quantity': 50, 'flags': False, 'token_symbol': None, 'strike_price': None, 'option_type': None}
```

## 8.5 `get_trade_message()`

Returns only trade messages as Python dictionaries.

Parameters: none.

Returns:

- `list[dict]`

Example:

```python
trades = reader.get_trade_message()

print("Trade count:", len(trades))
print(trades[0])
```

Expected output:

```text
Trade count: 70000
{'message_kind': 'trade', 'seq_no': 99, 'msg_len': 48, 'stream_id': 2, 'msg_type': 'T', 'exch_ts': 1716269402123456789, 'local_ts': 1716269403123456789, 'buy_order_id': 111111111, 'sell_order_id': 222222222, 'token': 40434, 'trade_price': 2195100, 'trade_quantity': 50, 'flags': False, 'token_symbol': None, 'strike_price': None, 'option_type': None}
```

## 8.6 `get_all_trade_message()`

Alias for `get_trade_message()`.

Example:

```python
trades = reader.get_all_trade_message()
print(len(trades))
```

Expected output:

```text
70000
```

## 8.7 `get_cache_summary()`

Returns a summary of the currently cached file.

Parameters: none.

Returns:

- `dict` with `file_source`, `total_messages`, `total_orders`, `total_trades`, and `memory_usage_bytes`.

Example:

```python
summary = reader.get_cache_summary()
print(summary)
```

Expected output:

```text
{'file_source': '/data/NSE_FO/Feed_FO_StreamID_1_21_05_2026.bin', 'total_messages': 1250000, 'total_orders': 1180000, 'total_trades': 70000, 'memory_usage_bytes': 80000000}
```

---

# 9. `StreamingBinaryLoader`

`StreamingBinaryLoader` reads one message at a time. Use it for large files, real-time-style processing, or when you do not want to hold the complete file in memory.

## 9.1 Create a streaming reader

```python
from fastreader import StreamingBinaryLoader

stream = StreamingBinaryLoader()
print(stream.is_end_of_msg())
```

Expected output:

```text
True
```

A new unopened stream reports end-of-message as `True` because no file is currently open.

## 9.2 `open_stream(file_path, count_messages=True)`

Opens a binary feed file for sequential reading.

Parameters:

| Parameter | Type | Required | Default | Description |
|---|---:|---:|---:|---|
| `file_path` | `str` | Yes | - | Binary feed path. |
| `count_messages` | `bool` | No | `True` | If true, scans the file to count messages before reading. For very large files, use `False` for faster startup. |

Returns:

- `int`: total message count if `count_messages=True`; otherwise `0`.

Example:

```python
from fastreader import StreamingBinaryLoader

feed_path = "/data/NSE_FO/Feed_FO_StreamID_1_21_05_2026.bin"

stream = StreamingBinaryLoader()
count = stream.open_stream(feed_path, count_messages=True)
print("Messages in stream:", count)
```

Expected output:

```text
Messages in stream: 1250000
```

Fast open without counting:

```python
stream = StreamingBinaryLoader()
count = stream.open_stream(feed_path, count_messages=False)
print(count)
```

Expected output:

```text
0
```

## 9.3 `get_next_msg()`

Reads the next available message and returns it as a Python dictionary. Returns `None` at EOF.

Parameters: none.

Returns:

- `dict | None`

Example:

```python
msg = stream.get_next_msg()
print(msg)
```

Expected output:

```text
{'message_kind': 'order', 'seq_no': 42, 'msg_len': 48, 'stream_id': 2, 'msg_type': 'N', 'exch_ts': 1716269400123456789, 'local_ts': 1716269401123456789, 'order_id': 12345678901, 'token': 40434, 'order_type': 'B', 'price': 2195050, 'quantity': 50, 'flags': False, 'token_symbol': None, 'strike_price': None, 'option_type': None}
```

Read in a loop:

```python
stream.reset_cursor()

while not stream.is_end_of_msg():
    msg = stream.get_next_msg()
    if msg is None:
        break
    print(msg["seq_no"], msg["msg_type"], msg["token"])
```

Expected output:

```text
42 N 40434
43 M 40434
44 T 40434
```

## 9.4 `is_end_of_msg()`

Checks whether the stream cursor is at the end of the file. It peeks ahead and then restores the current cursor position.

Parameters: none.

Returns:

- `bool`: `True` if there is no next message, otherwise `False`.

Example:

```python
stream.reset_cursor()

print("Before reading:", stream.is_end_of_msg())
first = stream.get_next_msg()
print("After one read:", stream.is_end_of_msg())
```

Expected output:

```text
Before reading: False
After one read: False
```

At the end:

```python
while stream.get_next_msg() is not None:
    pass

print(stream.is_end_of_msg())
```

Expected output:

```text
True
```

## 9.5 `reset_cursor()`

Moves the stream cursor back to the beginning of the opened file.

Parameters: none.

Returns:

- `None`

Example:

```python
stream.get_next_msg()
stream.get_next_msg()

stream.reset_cursor()
first_msg_again = stream.get_next_msg()

print(first_msg_again["seq_no"])
```

Expected output:

```text
42
```

## 9.6 `attach_symbol_master(master)`

Attaches a loaded `SymbolMaster`. After this, every `get_next_msg()` result is automatically enriched if the token exists in the symbol master.

Parameters:

| Parameter | Type | Required | Description |
|---|---:|---:|---|
| `master` | `SymbolMaster` | Yes | Loaded symbol master object. |

Returns:

- `None`

Example:

```python
from fastreader import StreamingBinaryLoader, SymbolMaster

feed_path = "/data/NSE_FO/Feed_FO_StreamID_1_21_05_2026.bin"
symbol_csv = "/data/CONTRACT/21_05_2026/NSE_FO_contract_21052026.csv"

sm = SymbolMaster()
sm.load(symbol_csv)

stream = StreamingBinaryLoader()
stream.open_stream(feed_path, count_messages=False)
stream.attach_symbol_master(sm)

msg = stream.get_next_msg()
print(msg["token"], msg["token_symbol"], msg["strike_price"], msg["option_type"], msg["expiry"], msg["lot_size"], msg["name"])
```

Expected output:

```text
40434 NIFTY 21950 CE 30-May-2024 50 NIFTY24MAY21950CE
```

## 9.7 `detach_symbol_master()`

Removes the attached symbol master. Future messages will again have `token_symbol`, `strike_price`, and `option_type` as `None` unless manually enriched.

Example:

```python
stream.detach_symbol_master()
msg = stream.get_next_msg()
print(msg.get("token_symbol"), msg.get("strike_price"), msg.get("option_type"))
```

Expected output:

```text
None None None
```

---

# 10. `SymbolMaster`

`SymbolMaster` loads NSE FO/CM contract master CSV data and gives fast token-to-contract lookups.

## 10.1 Create a symbol master

```python
from fastreader import SymbolMaster

sm = SymbolMaster()
print(sm)
print(len(sm))
```

Expected output:

```text
SymbolMaster(contracts=0)
0
```

## 10.2 `load(csv_path)`

Loads an explicit contract master CSV file.

Parameters:

| Parameter | Type | Required | Description |
|---|---:|---:|---|
| `csv_path` | `str` | Yes | Full path to the NSE contract master CSV. |

Returns:

- `int`: number of contracts loaded.

Possible errors:

- CSV file does not exist.
- Required columns are missing.
- File cannot be read.

Example:

```python
from fastreader import SymbolMaster

symbol_csv = "/data/CONTRACT/21_05_2026/NSE_FO_contract_21052026.csv"

sm = SymbolMaster()
count = sm.load(symbol_csv)

print("Contracts loaded:", count)
print(sm)
```

Expected output:

```text
Contracts loaded: 95632
SymbolMaster(contracts=95632)
```

## 10.3 `load_for_date(segment, day, month, year, base_path=None)`

Builds the standard NSE contract master path and loads the CSV.

Path pattern:

```text
{base_path}/CONTRACT/{DD}_{MM}_{YYYY}/NSE_{FO|CM}_contract_{DD}{MM}{YYYY}.csv
```

Parameters:

| Parameter | Type | Required | Default | Description |
|---|---:|---:|---:|---|
| `segment` | `str` | Yes | - | `NSE_FO`, `FO`, `NSE_CM`, or `CM`. |
| `day` | `int` | Yes | - | Day of month. |
| `month` | `int` | Yes | - | Month number. |
| `year` | `int` | Yes | - | Four-digit year. |
| `base_path` | `str | None` | No | `/nas/50.30` | Root path. |

Returns:

- `int`: number of contracts loaded.

Example:

```python
sm = SymbolMaster()
count = sm.load_for_date("NSE_FO", day=21, month=5, year=2026, base_path="/data")
print(count)
```

Expected output:

```text
95632
```

Possible error:

```text
RuntimeError: unknown segment 'BSE_FO' - expected NSE_FO, FO, NSE_CM, or CM
```

## 10.4 `lookup(token)`

Looks up a single token.

Parameters:

| Parameter | Type | Required | Description |
|---|---:|---:|---|
| `token` | `int` | Yes | Instrument token. |

Returns:

- `dict` with `found=True` and metadata if token exists.
- `dict` with `found=False` and `None` fields if token is unknown.

Example:

```python
info = sm.lookup(40434)
print(info)
```

Expected output:

```text
{'token': 40434, 'found': True, 'symbol': 'NIFTY', 'name': 'NIFTY24MAY21950CE', 'option_type': 'CE', 'strike': 21950, 'expiry': '30-May-2024', 'lot_size': 50}
```

Unknown token:

```python
print(sm.lookup(999999999))
```

Expected output:

```text
{'token': 999999999, 'found': False, 'symbol': None, 'name': None, 'option_type': None, 'strike': None, 'expiry': None, 'lot_size': None}
```

## 10.5 `enrich(msg)`

Enriches a message dictionary in place. Use it when you already have a message from `get_next_msg()` and want to add contract metadata manually.

Parameters:

| Parameter | Type | Required | Description |
|---|---:|---:|---|
| `msg` | `dict` | Yes | Message dictionary returned by `StreamingBinaryLoader.get_next_msg()`. |

Returns:

- `bool`: `True` if token was found and metadata was added, otherwise `False`.

Example:

```python
stream.reset_cursor()
msg = stream.get_next_msg()

found = sm.enrich(msg)

print("Found:", found)
print(msg["token_symbol"], msg["strike_price"], msg["option_type"], msg["expiry"], msg["lot_size"], msg["name"])
```

Expected output:

```text
Found: True
NIFTY 21950 CE 30-May-2024 50 NIFTY24MAY21950CE
```

---

# 11. `OrderbookBuilder`

`OrderbookBuilder` builds and queries token-wise order book state from decoded feed messages.

Use it when you want:

- Active tokens.
- Full bid/ask depth.
- Top N levels.
- CSV snapshot rows.
- Message type filters.
- Incremental streaming order-book updates.

## 11.1 Create an order book builder

```python
from fastreader import OrderbookBuilder

builder = OrderbookBuilder()
print(builder.get_active_tokens())
```

Expected output:

```text
[]
```

## 11.2 `apply_filter(logic_criteria=None)`

Restricts which message types are processed by the builder.

Parameters:

| Parameter | Type | Required | Description |
|---|---:|---:|---|
| `logic_criteria` | `list[str] | None` | No | Message types to allow, for example `["N", "M", "X"]`. Use `None` to clear the filter. |

Returns:

- `None`

Example: process only new and modify order messages.

```python
builder = OrderbookBuilder()
builder.apply_filter(["N", "M"])

processed = builder.build_from_list(reader)
print(processed)
```

Expected output:

```text
950000
```

Clear filter:

```python
builder.apply_filter(None)
```

## 11.3 `build_from_list(source)`

Builds the order book from either:

- a `MessageCacheReader`, or
- a `list[dict]` of decoded messages.

Parameters:

| Parameter | Type | Required | Description |
|---|---:|---:|---|
| `source` | `MessageCacheReader | list[dict]` | Yes | Source messages. |

Returns:

- `int`: number of messages accepted and processed.

Example using `MessageCacheReader`:

```python
builder = OrderbookBuilder()
processed = builder.build_from_list(reader)
print("Processed:", processed)
```

Expected output:

```text
Processed: 1180000
```

Example using a message list:

```python
orders = reader.get_order_message()

builder = OrderbookBuilder()
processed = builder.build_from_list(orders)

print(processed)
```

Expected output:

```text
1180000
```

Possible errors:

- Source is neither `MessageCacheReader` nor `list[dict]`.
- Message dictionary is missing required keys.
- Unsupported `msg_type`.

## 11.4 `build_from_source(source, limit=None)`

Builds the order book from either a cached reader or a streaming reader.

Parameters:

| Parameter | Type | Required | Default | Description |
|---|---:|---:|---:|---|
| `source` | `MessageCacheReader | StreamingBinaryLoader` | Yes | Source reader. |
| `limit` | `int | None` | No | `None` | Maximum number of accepted messages to process from a streaming reader. |

Returns:

- `int`: number of accepted messages processed.

Example from cache:

```python
builder = OrderbookBuilder()
processed = builder.build_from_source(reader)
print(processed)
```

Expected output:

```text
1180000
```

Example from stream:

```python
stream = StreamingBinaryLoader()
stream.open_stream(feed_path, count_messages=False)

builder = OrderbookBuilder()
processed = builder.build_from_source(stream, limit=10000)

print(processed)
```

Expected output:

```text
10000
```

## 11.5 `orderbook_add_msg(msg)`

Adds one decoded message dictionary into the order book. Use this for step-by-step streaming processing.

Parameters:

| Parameter | Type | Required | Description |
|---|---:|---:|---|
| `msg` | `dict` | Yes | One message returned by `get_next_msg()`. |

Returns:

- `bool`: `True` if accepted and applied; `False` if skipped by filters or business rules.

Example:

```python
stream.reset_cursor()
builder = OrderbookBuilder()

while not stream.is_end_of_msg():
    msg = stream.get_next_msg()
    if msg is None:
        break
    accepted = builder.orderbook_add_msg(msg)
    if accepted:
        token = msg["token"]
        snapshot = builder.get_snapshot(token, levels=5)
        print(snapshot)
        break
```

Expected output:

```text
{'token': 40434, 'found': True, 'mid_price': 2195050, 'best_bid': (2195050, 50), 'best_ask': None, 'spread': None, 'bids': [(2195050, 50)], 'asks': []}
```

## 11.6 `get_active_tokens()`

Returns tokens currently present in the order book.

Parameters: none.

Returns:

- `list[int]`

Example:

```python
active_tokens = builder.get_active_tokens()
print(active_tokens[:10])
```

Expected output:

```text
[40434, 40435, 40436, 50120, 50121]
```

## 11.7 `get_full_depth(token)`

Returns complete bid/ask depth for a token.

Parameters:

| Parameter | Type | Required | Description |
|---|---:|---:|---|
| `token` | `int` | Yes | Instrument token. |

Returns:

- `dict` with token, found flag, best bid, best ask, spread, bids, and asks.

Example:

```python
depth = builder.get_full_depth(40434)
print(depth)
```

Expected output:

```text
{'token': 40434, 'found': True, 'best_bid': (2195050, 100), 'best_ask': (2195100, 50), 'spread': 50, 'bids': [(2195050, 100), (2195000, 250)], 'asks': [(2195100, 50), (2195150, 300)]}
```

Unknown token:

```python
print(builder.get_full_depth(999999999))
```

Expected output:

```text
{'token': 999999999, 'found': False, 'best_bid': None, 'best_ask': None, 'spread': None, 'bids': [], 'asks': []}
```

## 11.8 `get_snapshot(token, levels=None)`

Returns top N bid/ask levels for one token. Default depth is 5 levels.

Parameters:

| Parameter | Type | Required | Default | Description |
|---|---:|---:|---:|---|
| `token` | `int` | Yes | - | Instrument token. |
| `levels` | `int | None` | No | `5` | Number of top levels. |

Returns:

- `dict` with token, found flag, mid price, best bid, best ask, spread, bids, and asks.

Example:

```python
snapshot = builder.get_snapshot(40434, levels=5)
print(snapshot)
```

Expected output:

```text
{'token': 40434, 'found': True, 'mid_price': 2195075, 'best_bid': (2195050, 100), 'best_ask': (2195100, 50), 'spread': 50, 'bids': [(2195050, 100), (2195000, 250)], 'asks': [(2195100, 50), (2195150, 300)]}
```

Unknown token:

```python
print(builder.get_snapshot(999999999, levels=5))
```

Expected output:

```text
{'token': 999999999, 'found': False, 'mid_price': 0, 'best_bid': None, 'best_ask': None, 'spread': None, 'bids': [], 'asks': []}
```

## 11.9 `snapshot_header()`

Returns the CSV header for a 5-level snapshot row.

Parameters: none.

Returns:

- `str`

Example:

```python
header = builder.snapshot_header()
print(header)
```

Expected output:

```text
local_ts,exch_ts,mid_price,bid_price_0,bid_qty_0,ask_price_0,ask_qty_0,bid_price_1,bid_qty_1,ask_price_1,ask_qty_1,bid_price_2,bid_qty_2,ask_price_2,ask_qty_2,bid_price_3,bid_qty_3,ask_price_3,ask_qty_3,bid_price_4,bid_qty_4,ask_price_4,ask_qty_4
```

## 11.10 `get_snapshot_row(token, levels=None)`

Returns one CSV-formatted snapshot row. The current implementation emits 5 bid/ask levels and fills missing levels with zeros.

Parameters:

| Parameter | Type | Required | Default | Description |
|---|---:|---:|---:|---|
| `token` | `int` | Yes | - | Instrument token. |
| `levels` | `int | None` | No | `5` | Requested levels. |

Returns:

- `str`: CSV row.

Example:

```python
print(builder.snapshot_header())
print(builder.get_snapshot_row(40434, levels=5))
```

Expected output:

```text
local_ts,exch_ts,mid_price,bid_price_0,bid_qty_0,ask_price_0,ask_qty_0,bid_price_1,bid_qty_1,ask_price_1,ask_qty_1,bid_price_2,bid_qty_2,ask_price_2,ask_qty_2,bid_price_3,bid_qty_3,ask_price_3,ask_qty_3,bid_price_4,bid_qty_4,ask_price_4,ask_qty_4
0,0,2195075,2195050,100,2195100,50,2195000,250,2195150,300,0,0,0,0,0,0,0,0,0,0,0,0
```

---

# 12. `FeedPathBuilder`

`FeedPathBuilder` creates standard NSE feed file paths.

## 12.1 Create a builder

```python
from fastreader import FeedPathBuilder

path_builder = FeedPathBuilder()
print(path_builder)
```

Expected output:

```text
FeedPathBuilder()
```

## 12.2 `build(segment, stream_id, day, month, year, base_path=None)`

Builds a path string without checking whether the file exists.

Parameters:

| Parameter | Type | Required | Default | Description |
|---|---:|---:|---:|---|
| `segment` | `str` | Yes | - | `NSE_CM`, `CM`, `NSE_FO`, or `FO`. |
| `stream_id` | `int` | Yes | - | Positive stream id. |
| `day` | `int` | Yes | - | Day of month. |
| `month` | `int` | Yes | - | Month number. |
| `year` | `int` | Yes | - | Four-digit year. |
| `base_path` | `str | None` | No | `/nas/50.30` | Root directory. |

Returns:

- `str`: constructed file path.

Example:

```python
builder = FeedPathBuilder()
path = builder.build("NSE_CM", stream_id=2, day=29, month=12, year=2025)
print(path)
```

Expected output:

```text
/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin
```

Custom base path:

```python
path = builder.build("NSE_FO", stream_id=1, day=1, month=6, year=2026, base_path="/mnt/data")
print(path)
```

Expected output:

```text
/mnt/data/NSE_FO/Feed_FO_StreamID_1_1_6_2026.bin
```

## 12.3 `build_and_verify(segment, stream_id, day, month, year, base_path=None)`

Builds the path and verifies that the file exists on disk.

Returns:

- `str`: constructed file path if it exists.

Possible errors:

- Segment is unknown.
- Stream id or date is invalid.
- File does not exist.

Example:

```python
path = builder.build_and_verify("NSE_CM", stream_id=2, day=29, month=12, year=2025)
print(path)
```

Expected output:

```text
/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin
```

---

# 13. Complete Jupyter examples

## 13.1 Load a file, inspect orders and trades

```python
from fastreader import MessageCacheReader

feed_path = "/data/NSE_FO/Feed_FO_StreamID_1_21_05_2026.bin"

reader = MessageCacheReader()
count = reader.load_to_cache(feed_path)

orders = reader.get_order_message()
trades = reader.get_trade_message()
summary = reader.get_cache_summary()

print("Total:", count)
print("Orders:", len(orders))
print("Trades:", len(trades))
print("Summary:", summary)
print("First order:", orders[0] if orders else None)
print("First trade:", trades[0] if trades else None)
```

Expected output:

```text
Total: 1250000
Orders: 1180000
Trades: 70000
Summary: {'file_source': '/data/NSE_FO/Feed_FO_StreamID_1_21_05_2026.bin', 'total_messages': 1250000, 'total_orders': 1180000, 'total_trades': 70000, 'memory_usage_bytes': 80000000}
First order: {'message_kind': 'order', 'seq_no': 42, ...}
First trade: {'message_kind': 'trade', 'seq_no': 99, ...}
```

## 13.2 Stream messages one by one and detect EOF

```python
from fastreader import StreamingBinaryLoader

feed_path = "/data/NSE_FO/Feed_FO_StreamID_1_21_05_2026.bin"

stream = StreamingBinaryLoader()
stream.open_stream(feed_path, count_messages=False)

read_count = 0
while not stream.is_end_of_msg():
    msg = stream.get_next_msg()
    if msg is None:
        break
    read_count += 1
    if read_count <= 3:
        print(msg)

print("Read:", read_count)
print("EOF:", stream.is_end_of_msg())
```

Expected output:

```text
{'message_kind': 'order', 'seq_no': 42, ...}
{'message_kind': 'order', 'seq_no': 43, ...}
{'message_kind': 'trade', 'seq_no': 44, ...}
Read: 1250000
EOF: True
```

## 13.3 Enrich streamed messages with token metadata

```python
from fastreader import StreamingBinaryLoader, SymbolMaster

feed_path = "/data/NSE_FO/Feed_FO_StreamID_1_21_05_2026.bin"
symbol_csv = "/data/CONTRACT/21_05_2026/NSE_FO_contract_21052026.csv"

sm = SymbolMaster()
sm.load(symbol_csv)

stream = StreamingBinaryLoader()
stream.open_stream(feed_path, count_messages=False)
stream.attach_symbol_master(sm)

msg = stream.get_next_msg()
print({
    "token": msg["token"],
    "token_symbol": msg.get("token_symbol"),
    "strike_price": msg.get("strike_price"),
    "option_type": msg.get("option_type"),
    "expiry": msg.get("expiry"),
    "lot_size": msg.get("lot_size"),
    "name": msg.get("name"),
})
```

Expected output:

```text
{'token': 40434, 'token_symbol': 'NIFTY', 'strike_price': 21950, 'option_type': 'CE', 'expiry': '30-May-2024', 'lot_size': 50, 'name': 'NIFTY24MAY21950CE'}
```

## 13.4 Build order book from cached messages

```python
from fastreader import MessageCacheReader, OrderbookBuilder

reader = MessageCacheReader()
reader.load_to_cache(feed_path)

builder = OrderbookBuilder()
processed = builder.build_from_list(reader)

print("Processed:", processed)
print("Active tokens:", builder.get_active_tokens()[:10])
```

Expected output:

```text
Processed: 1180000
Active tokens: [40434, 40435, 40436, 50120, 50121]
```

## 13.5 Build order book from a streaming reader

```python
from fastreader import StreamingBinaryLoader, OrderbookBuilder

stream = StreamingBinaryLoader()
stream.open_stream(feed_path, count_messages=False)

builder = OrderbookBuilder()
processed = builder.build_from_source(stream, limit=50000)

print("Processed:", processed)
print("Active tokens:", builder.get_active_tokens()[:10])
```

Expected output:

```text
Processed: 50000
Active tokens: [40434, 40435, 40436]
```

## 13.6 Build incrementally and print snapshot after each accepted message

```python
stream.reset_cursor()
builder = OrderbookBuilder()

for _ in range(5):
    msg = stream.get_next_msg()
    if msg is None:
        break

    accepted = builder.orderbook_add_msg(msg)
    print("accepted:", accepted, "msg_type:", msg["msg_type"], "token:", msg["token"])

    if accepted:
        print(builder.get_snapshot(msg["token"], levels=5))
```

Expected output:

```text
accepted: True msg_type: N token: 40434
{'token': 40434, 'found': True, 'mid_price': 2195050, 'best_bid': (2195050, 50), 'best_ask': None, 'spread': None, 'bids': [(2195050, 50)], 'asks': []}
accepted: True msg_type: M token: 40434
{'token': 40434, 'found': True, 'mid_price': 2195075, 'best_bid': (2195050, 50), 'best_ask': (2195100, 50), 'spread': 50, 'bids': [(2195050, 50)], 'asks': [(2195100, 50)]}
```

---

# 14. Troubleshooting

## 14.1 Missing file path

Symptom:

```text
RuntimeError: No such file or directory
```

Cause:

- The binary feed path or CSV path does not exist.

Fix:

```python
from pathlib import Path

path = Path(feed_path)
print(path.exists(), path)
```

Use `FeedPathBuilder.build_and_verify()` when you want the library to validate the path for you.

## 14.2 Invalid binary header

Symptom:

```text
RuntimeError: invalid first message type: 65
```

Cause:

- The file is not a supported NSE binary feed file.
- The file starts with a message type other than `N`, `M`, `X`, or `T`.
- The file is compressed, text, or otherwise not raw binary feed data.

Fix:

- Confirm the file source.
- Confirm it is not zipped or partially copied.
- Test another known-good feed file.

## 14.3 Empty or incomplete file

Symptom:

```text
RuntimeError: file is empty or missing a complete message header
```

Cause:

- Empty file.
- File copy did not finish.
- Wrong path points to a placeholder file.

Fix:

```python
from pathlib import Path

p = Path(feed_path)
print(p.exists(), p.stat().st_size)
```

## 14.4 Truncated message header

Symptom:

```text
RuntimeError: truncated message header
```

Cause:

- The file ended before a full header could be read.

Fix:

- Recopy the source file.
- Compare file size with the original location.

## 14.5 Truncated message payload

Symptom:

```text
RuntimeError: truncated message payload
```

Cause:

- Header was present, but the complete packet body was missing.
- File is corrupted or partially copied.

Fix:

- Recopy the feed file.
- Validate the file was completely written before reading.

## 14.6 Missing symbol master CSV columns

Symptom:

```text
RuntimeError: column 'FinInstrmId' not found in /path/to/file.csv
```

Cause:

- The CSV is not the expected contract master file.
- Required columns are missing or renamed.

Required columns:

```text
FinInstrmId,TckrSymb,XpryDt,StrkPric,OptnTp,StockNm
```

Optional columns:

```text
NewBrdLotQty,MinLot
```

Fix:

```python
import pandas as pd

cols = pd.read_csv(symbol_csv, nrows=0).columns.tolist()
print(cols)
```

## 14.7 Unknown segment name

Symptom:

```text
RuntimeError: unknown segment 'BSE_FO' - expected NSE_FO, FO, NSE_CM, or CM
```

Cause:

- `load_for_date()` or `FeedPathBuilder` received an unsupported segment.

Fix:

Use one of:

```text
NSE_FO, FO, NSE_CM, CM
```

## 14.8 Unsupported message type

Symptom:

```text
TypeError: unsupported msg_type: A
```

Cause:

- You passed a custom dictionary into `OrderbookBuilder` with an unsupported `msg_type`.

Supported message types:

```text
N, M, X, T
```

Fix:

```python
for msg in messages:
    if msg["msg_type"] not in {"N", "M", "X", "T"}:
        print("Unsupported:", msg)
```

## 14.9 Empty order book snapshot

Symptom:

```python
{'token': 40434, 'found': False, 'mid_price': 0, 'best_bid': None, 'best_ask': None, 'spread': None, 'bids': [], 'asks': []}
```

Possible causes:

- The token has not received any accepted order-book messages yet.
- The builder was not populated.
- A filter skipped all relevant messages.
- You queried the wrong token.
- Only trade messages were processed, and no book levels were available.

Fix:

```python
print("Active tokens:", builder.get_active_tokens()[:20])
print("Cache summary:", reader.get_cache_summary())

builder.apply_filter(None)
processed = builder.build_from_list(reader)
print("Processed:", processed)
```

---

# 15. Best practices

1. Use `MessageCacheReader` for smaller files and repeated analysis.
2. Use `StreamingBinaryLoader` for very large files or incremental processing.
3. Use `count_messages=False` for faster stream startup on large files.
4. Always check `get_cache_summary()` after loading a file.
5. Use `SymbolMaster` when users need readable symbol, strike, option type, expiry, lot size, and contract name.
6. Attach `SymbolMaster` to `StreamingBinaryLoader` for automatic enrichment.
7. Use `apply_filter(["N", "M", "X"])` when you want to exclude trades from order-book processing.
8. Use `get_active_tokens()` before querying snapshots.
9. Treat prices as feed-scaled integer prices unless your application converts them to rupees/points.
10. Keep feed binary and contract master CSV from the same trading date whenever possible.

---

# 16. Quick API reference

## `MessageCacheReader`

| Method | Returns | Purpose |
|---|---:|---|
| `load_to_cache(file_path)` | `int` | Load all feed messages into memory. |
| `get_all_messages()` | `list[str]` | Get formatted string messages. |
| `get_order_message()` | `list[dict]` | Get only order messages. |
| `get_trade_message()` | `list[dict]` | Get only trade messages. |
| `get_all_trade_message()` | `list[dict]` | Alias for trade messages. |
| `get_cache_summary()` | `dict` | Get file source, counts, and memory usage. |

## `StreamingBinaryLoader`

| Method | Returns | Purpose |
|---|---:|---|
| `open_stream(file_path, count_messages=True)` | `int` | Open a file for sequential reading. |
| `get_next_msg()` | `dict | None` | Read next message. |
| `is_end_of_msg()` | `bool` | Check EOF without advancing cursor. |
| `reset_cursor()` | `None` | Seek back to file start. |
| `attach_symbol_master(master)` | `None` | Enable auto-enrichment. |
| `detach_symbol_master()` | `None` | Disable auto-enrichment. |

## `SymbolMaster`

| Method | Returns | Purpose |
|---|---:|---|
| `load(csv_path)` | `int` | Load contract master from explicit path. |
| `load_for_date(segment, day, month, year, base_path=None)` | `int` | Build standard contract path and load. |
| `lookup(token)` | `dict` | Get metadata for token. |
| `enrich(msg)` | `bool` | Enrich one message dictionary in place. |
| `len(sm)` | `int` | Number of loaded contracts. |
| `repr(sm)` | `str` | Display contract count. |

## `OrderbookBuilder`

| Method | Returns | Purpose |
|---|---:|---|
| `apply_filter(logic_criteria=None)` | `None` | Restrict processed message types. |
| `build_from_list(source)` | `int` | Build from cache or list of dicts. |
| `build_from_source(source, limit=None)` | `int` | Build from cache or stream. |
| `orderbook_add_msg(msg)` | `bool` | Add one decoded message. |
| `get_active_tokens()` | `list[int]` | Tokens with active book state. |
| `get_full_depth(token)` | `dict` | Full bid/ask depth. |
| `get_snapshot(token, levels=None)` | `dict` | Top N levels. |
| `snapshot_header()` | `str` | CSV snapshot header. |
| `get_snapshot_row(token, levels=None)` | `str` | CSV snapshot row. |

## `FeedPathBuilder`

| Method | Returns | Purpose |
|---|---:|---|
| `build(segment, stream_id, day, month, year, base_path=None)` | `str` | Build feed file path. |
| `build_and_verify(segment, stream_id, day, month, year, base_path=None)` | `str` | Build path and verify file exists. |

---

# 17. Minimal end-to-end example

```python
from fastreader import FeedPathBuilder, SymbolMaster, StreamingBinaryLoader, OrderbookBuilder

base_path = "/data"
day, month, year = 21, 5, 2026
stream_id = 1
segment = "NSE_FO"

# Build paths
path_builder = FeedPathBuilder()
feed_path = path_builder.build(segment, stream_id, day, month, year, base_path=base_path)

# Load symbol master
sm = SymbolMaster()
sm.load_for_date(segment, day, month, year, base_path=base_path)

# Stream binary feed
stream = StreamingBinaryLoader()
stream.open_stream(feed_path, count_messages=False)
stream.attach_symbol_master(sm)

# Build order book from first 100000 accepted messages
book = OrderbookBuilder()
processed = book.build_from_source(stream, limit=100000)

print("Processed:", processed)
print("Active tokens:", book.get_active_tokens()[:10])

# Query first active token
active = book.get_active_tokens()
if active:
    token = active[0]
    print("Token info:", sm.lookup(token))
    print("Snapshot:", book.get_snapshot(token, levels=5))
    print("CSV header:", book.snapshot_header())
    print("CSV row:", book.get_snapshot_row(token, levels=5))
else:
    print("No active tokens found. Check feed file, message filters, and token date.")
```

Expected output:

```text
Processed: 100000
Active tokens: [40434, 40435, 40436, 50120, 50121]
Token info: {'token': 40434, 'found': True, 'symbol': 'NIFTY', 'name': 'NIFTY24MAY21950CE', 'option_type': 'CE', 'strike': 21950, 'expiry': '30-May-2024', 'lot_size': 50}
Snapshot: {'token': 40434, 'found': True, 'mid_price': 2195075, 'best_bid': (2195050, 100), 'best_ask': (2195100, 50), 'spread': 50, 'bids': [(2195050, 100)], 'asks': [(2195100, 50)]}
CSV header: local_ts,exch_ts,mid_price,bid_price_0,bid_qty_0,ask_price_0,ask_qty_0,...
CSV row: 0,0,2195075,2195050,100,2195100,50,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
```

---
