Metadata-Version: 2.4
Name: hft-lob
Version: 0.2.9
Summary: Python wrapper for a high-performance Rust orderbook CLI
Home-page: https://github.com/pratima/hft_lob
Author: Pratima Kumari
Author-email: Pratima Kumari <pratimarmoney@gmail.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/pratima/hft_lob
Requires-Python: >=3.7
Description-Content-Type: text/markdown
Dynamic: author
Dynamic: home-page
Dynamic: requires-python

# hft-lob

[![PyPI version](https://img.shields.io/pypi/v/hft-lob)](https://pypi.org/project/hft-lob/)
[![Python](https://img.shields.io/pypi/pyversions/hft-lob)](https://pypi.org/project/hft-lob/)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
[![Platform](https://img.shields.io/badge/platform-linux--x86__64-lightgrey)](https://pypi.org/project/hft-lob/)

High-performance Python library for reading NSE binary market feed files and reconstructing a full 5-level Limit Order Book (LOB). Powered by a compiled Rust binary — zero Python overhead on the critical path.

---

## Features

- Reconstruct LOB from NSE CM binary feed files
- Support for single or multiple instrument tokens
- CLI commands — no arguments, no paths, no tokens on the command line
- `get_bid` / `get_ask` commands — instantly see bid/ask levels of the current row
- `LOBRow` class — named field access in Python (`row.bid(0)`, `row.all_asks()`)
- Cache system — Rust binary runs once, every subsequent call reads from disk instantly
- 23-field CSV output per tick (timestamps, mid-price, 5-level bid/ask)
- No dependencies — Rust binary is bundled

---

## Architecture

```
hft-lob
├── hft_lob/
│   ├── cli.py            # LOBRow class + Reader class + CLI entry point
│   └── bin/
│       └── orderbook-linux-x86_64   # Compiled Rust binary
~/.hft_lob                # User config  (FILE + TOKEN)
~/.hft_lob_cache          # Message cache (built once, reused)
~/.hft_lob_state          # Read cursor   (INDEX + FILE + TOKEN)
```

**Data flow:**

```
NSE .bin feed file
       │
       ▼
 Rust binary (subprocess)          ← runs ONCE, then result is cached
  orderbook-linux-x86_64
       │
       ▼
  ~/.hft_lob_cache                 ← all CSV rows persisted to disk
       │
       ├──── ~/.hft_lob_state      ← tracks current INDEX
       │
    ┌──┴───────────────────────────────────────┐
    ▼                                          ▼
hft-lob get_next                   hft-lob get_all
(reads 1 row, advances INDEX)      (streams full cache to stdout)
    │
    ├── hft-lob get_bid [level]    ← bid price+qty of current row
    └── hft-lob get_ask [level]    ← ask price+qty of current row
```

The Rust binary runs **once per unique FILE+TOKEN combination**. All subsequent calls read directly from `~/.hft_lob_cache` — microseconds instead of seconds.

---

## Install

```bash
pip install hft-lob
```

---

## Configure (once)

Create `~/.hft_lob` — the only setup you ever need:

```bash
printf 'FILE=/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin\nTOKEN=1333,2885,5900\n' > ~/.hft_lob
```

| Key | Description |
|-----|-------------|
| `FILE` | Absolute path to the NSE binary feed file |
| `TOKEN` | Instrument token(s). Comma-separated for multiple. |

---

## CLI Usage

### Navigate rows

```bash
hft-lob get_next
```
Advances to the next row and prints all 23 fields with names:
```
--- row 0 ---
  local_ts: 1766979473076306016
  exch_ts: 1451466473069645718
  mid_price: 0
  bid_price_0: 99310
  bid_qty_0: 13456
  ask_price_0: 0
  ask_qty_0: 0
  ...
```

### Inspect bid levels

```bash
hft-lob get_bid          # all 5 bid levels of current row
```
```
bid levels:
  [0]  price=99310  qty=13456
  [1]  price=99300  qty=5200
  [2]  price=99290  qty=8100
  [3]  price=0      qty=0
  [4]  price=0      qty=0
```

```bash
hft-lob get_bid 0        # best bid only
```
```
bid level 0:  price=99310  qty=13456
```

### Inspect ask levels

```bash
hft-lob get_ask          # all 5 ask levels
hft-lob get_ask 0        # best ask only
hft-lob get_ask 2        # ask level 2 only
```

### Other commands

```bash
hft-lob get_all          # print every row as raw CSV (for piping/saving)
hft-lob eof              # True if all rows consumed, False otherwise
hft-lob reset            # go back to row 0 (cache is kept)
```

**Pipe examples:**

```bash
hft-lob get_all | wc -l          # count total rows
hft-lob get_all | head -6        # preview first 5 rows
hft-lob get_all > lob_data.csv   # save to file
```

---

## Python API

### `LOBRow` — named field access

```python
from hft_lob.cli import LOBRow

row = LOBRow("1766979473076306016,1451466473069645718,0,99310,13456,...")

# Best bid and ask
price, qty = row.bid(0)     # bid level 0
price, qty = row.ask(0)     # ask level 0

# All 5 levels
row.all_bids()   # [(price, qty), (price, qty), ...]
row.all_asks()   # [(price, qty), (price, qty), ...]

# Direct field access
row.mid_price
row.bid_price_0
row.ask_qty_2
row.local_ts
row.exch_ts

# As dict
row.to_dict()    # {'local_ts': '...', 'bid_price_0': '...', ...}
```

### `Reader` — bulk processing

```python
from hft_lob.cli import Reader

r = Reader("/path/to/feed.bin", tokens=[1333, 2885, 5900])

r.get_next_message()    # returns one CSV string, or None
r.get_all_messages()    # returns list of all CSV strings
r.is_end_of_file()      # returns True / False
r.header                # CSV column names string
```

---

## CSV Output Format

23 fields per row:

| Field | Description |
|-------|-------------|
| `local_ts` | Local timestamp (nanoseconds epoch) |
| `exch_ts` | Exchange timestamp (nanoseconds epoch) |
| `mid_price` | `(best_bid + best_ask) / 2` |
| `bid_price_0` – `bid_price_4` | Bid price at depth levels 0–4 |
| `bid_qty_0` – `bid_qty_4` | Bid quantity at depth levels 0–4 |
| `ask_price_0` – `ask_price_4` | Ask price at depth levels 0–4 |
| `ask_qty_0` – `ask_qty_4` | Ask quantity at depth levels 0–4 |

---

## Load into pandas

```python
import io
import pandas as pd
from hft_lob.cli import Reader

r = Reader("/path/to/feed.bin", tokens=[1333, 2885, 5900])
msgs = r.get_all_messages()

df = pd.read_csv(io.StringIO(r.header + "\n" + "\n".join(msgs)))
df["exch_ts"] = pd.to_datetime(df["exch_ts"], unit="ns")
print(df.head())
print(f"Total rows: {len(df)}")
```

---

## Requirements

| Item | Requirement |
|------|-------------|
| OS | Linux x86_64 |
| Python | 3.7+ |
| Dependencies | None |

The Rust binary (`orderbook-linux-x86_64`) is bundled — no separate install, no Rust toolchain needed.

---

## License

MIT

---

## Features

- Reconstruct LOB from NSE CM binary feed files
- Support for single or multiple instrument tokens
- Three simple CLI commands — no arguments, no paths, no tokens on the command line
- Clean Python `Reader` API for scripting and backtesting
- 23-field CSV output per tick (timestamps, mid-price, 5-level bid/ask)
- No dependencies — Rust binary is bundled

---

## Architecture

```
hft-lob
├── hft_lob/
│   ├── cli.py            # Reader class + CLI entry point
│   └── bin/
│       └── orderbook-linux-x86_64   # Compiled Rust binary
~/.hft_lob                # User config  (FILE + TOKEN)
~/.hft_lob_cache          # Message cache (built once, reused)
~/.hft_lob_state          # Read cursor   (INDEX + FILE + TOKEN)
```

**Data flow:**

```
NSE .bin feed file
       │
       ▼
 Rust binary (subprocess)          ← runs ONCE, then result is cached
  orderbook-linux-x86_64
       │
       ▼
  ~/.hft_lob_cache                 ← all CSV rows persisted to disk
       │
       ├──── ~/.hft_lob_state      ← tracks current INDEX
       │
    ┌──┴──────────────────────┐
    ▼                         ▼
 hft-lob get_next        hft-lob get_all
 (reads 1 row,           (streams full
  advances INDEX)         cache to stdout)
```

The Rust binary runs **once per unique FILE+TOKEN combination**. Subsequent `get_next` calls read directly from the disk cache — microseconds instead of seconds.

---

## Install

```bash
pip install hft-lob
```

---

## Configure (once)

Create `~/.hft_lob` — this is the only setup you ever need to do:

```bash
cat > ~/.hft_lob << 'EOF'
FILE=/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin
TOKEN=1333,2885,5900
EOF
```

| Key | Description |
|-----|-------------|
| `FILE` | Absolute path to the NSE binary feed file |
| `TOKEN` | Instrument token(s). Comma-separated for multiple. |

---

## CLI Usage

No arguments. No file path. No token. Just run:

```bash
hft-lob get_next
```
Prints the next LOB tick as a single CSV row.

```bash
hft-lob get_all
```
Prints every LOB tick, one CSV row per line (includes header on first line).

```bash
hft-lob eof
```
Prints `True` if all messages consumed, `False` if more remain.

```bash
hft-lob reset
```
Resets the read cursor back to the first message (cache is kept).

**Pipe examples:**

```bash
# Count total messages
hft-lob get_all | wc -l

# Preview first 5 rows
hft-lob get_all | head -6

# Save to CSV
hft-lob get_all > lob_data.csv
```

---

## Python API

```python
from hft_lob.cli import Reader

# Single token
r = Reader("/path/to/feed.bin", tokens=1333)

# Multiple tokens — merged into one stream
r = Reader("/path/to/feed.bin", tokens=[1333, 2885, 5900])
```

| Method / Attribute | Returns | Description |
|--------------------|---------|-------------|
| `r.get_next_message()` | `str \| None` | Next CSV row, or `None` at EOF |
| `r.get_all_messages()` | `list[str]` | All CSV rows as a list |
| `r.is_end_of_file()` | `bool` | `True` after all messages are consumed |
| `r.header` | `str` | Comma-separated column names |

**Streaming pattern:**

```python
r = Reader("/path/to/feed.bin", tokens=1333)
while not r.is_end_of_file():
    row = r.get_next_message()
    if row:
        print(row)
```

---

## CSV Output Format

23 fields per row:

| Field | Description |
|-------|-------------|
| `local_ts` | Local timestamp (nanoseconds epoch) |
| `exch_ts` | Exchange timestamp (nanoseconds epoch) |
| `mid_price` | `(best_bid + best_ask) / 2` |
| `bid_price_0` – `bid_price_4` | Bid price at depth levels 0–4 |
| `bid_qty_0` – `bid_qty_4` | Bid quantity at depth levels 0–4 |
| `ask_price_0` – `ask_price_4` | Ask price at depth levels 0–4 |
| `ask_qty_0` – `ask_qty_4` | Ask quantity at depth levels 0–4 |

---

## Load into pandas

```python
import io
import pandas as pd
from hft_lob.cli import Reader

r = Reader("/path/to/feed.bin", tokens=[1333, 2885, 5900])
msgs = r.get_all_messages()

df = pd.read_csv(io.StringIO(r.header + "\n" + "\n".join(msgs)))
df["exch_ts"] = pd.to_datetime(df["exch_ts"], unit="ns")
print(df.head())
print(f"Total rows: {len(df)}")
```

---

## Requirements

| Item | Requirement |
|------|-------------|
| OS | Linux x86_64 |
| Python | 3.7+ |
| Dependencies | None |

The Rust binary (`orderbook-linux-x86_64`) is bundled inside the package — no separate install, no Rust toolchain needed.

---

## License

MIT
