Metadata-Version: 2.4
Name: OrderPulse
Version: 0.2.11
Summary: High-performance exchange feed parser and orderflow analytics engine with Rust and Python bindings
Requires-Python: >=3.9
Description-Content-Type: text/markdown

# FastReader

High-performance binary market-data parser built in Rust with Python bindings using PyO3.

FastReader is designed for low-latency parsing and sequential processing of binary order and trade packets. It provides efficient market-data extraction, order book generation, top-level market depth calculation, and Python interoperability.

---

# Table of Contents

1. Overview
2. Key Features
3. Architecture
4. Project Structure
5. Core Components
6. Data Flow
7. Installation
8. Build Instructions
9. Python Integration
10. API Reference
11. Usage Examples
12. Orderbook Engine
13. Performance Considerations
14. Error Handling
15. Example Output
16. Recommended Improvements
17. Future Enhancements
18. License

---

# Overview

FastReader is a Rust-powered binary parser optimized for market-data systems where throughput and deterministic processing matter.

The library:

* Parses binary order packets
* Parses binary trade packets
* Supports sequential message streaming
* Generates real-time orderbook snapshots
* Calculates mid-price dynamically
* Exposes Rust functionality to Python through PyO3

The project follows a modular architecture for maintainability and scalability.

---

# Key Features

## High Performance Parsing

Uses Rust memory safety and zero-cost abstractions for fast packet decoding.

## Sequential Message Processing

Supports message-by-message iteration using internal indexing.

## Python Bindings

Exposes Rust backend directly to Python.

## Token Filtering

Allows filtering by instrument token during initialization.

## Orderbook Reconstruction

Maintains orderbook state and generates top bid/ask levels.

## Mid Price Calculation

Calculates real-time midpoint from best bid and ask.

## Modular Design

Separated parsing, structures, and orderbook logic.

---

# Architecture

```text
                    +----------------------+
                    |   Binary Market Data |
                    |      (.bin file)     |
                    +----------+-----------+
                               |
                               v
                +----------------------------+
                |  read_trd_ord_only.rs      |
                |  Binary Packet Parser      |
                +-------------+--------------+
                              |
                              v
                +----------------------------+
                |       structure.rs         |
                |  Message / Packet Models   |
                +-------------+--------------+
                              |
                              v
                +----------------------------+
                |          lib.rs            |
                |  Python Interface Layer    |
                |  Sequential Processing     |
                +-------------+--------------+
                              |
          +-------------------+-------------------+
          |                                       |
          v                                       v
+---------------------+               +----------------------+
| Message Formatting  |               | orderbook.rs         |
| Human Readable View |               | OrderBook Engine     |
+---------------------+               +-----------+----------+
                                                  |
                                                  v
                                   +----------------------------+
                                   | Mid Price + Top 5 Levels   |
                                   +----------------------------+
                                                  |
                                                  v
                                   +----------------------------+
                                   | Python Consumer / Strategy |
                                   +----------------------------+
```

---

# Project Structure

```text
fastreader/
│
├── src/
│   ├── lib.rs
│   ├── structure.rs
│   ├── read_trd_ord_only.rs
│   └── orderbook.rs
│
├── Cargo.toml
├── pyproject.toml
└── README.md
```

---

# Core Components

# 1. lib.rs

Main library entry point.

Responsibilities:

* Python module registration
* Expose Rust APIs to Python
* Message formatting
* Sequential processing
* Orderbook integration

Key Objects:

```rust
ReadMsgFromBinary
```

Main public interface exposed to Python.

---

# 2. structure.rs

Defines all packet structures.

Typical responsibilities:

* Binary packet layout
* Message enums
* Header structures
* Order packet structures
* Trade packet structures

Example:

```rust
pub enum Message {
    Order(OrderPacket),
    Trade(TradePacket),
}
```

---

# 3. read_trd_ord_only.rs

Binary parser implementation.

Responsibilities:

* Read binary file
* Deserialize packets
* Detect message type
* Construct Message enums

Core Function:

```rust
read_messages(path)
```

Returns:

```rust
Vec<Message>
```

---

# 4. orderbook.rs

Maintains live orderbook state.

Responsibilities:

* Process order updates
* Process trade updates
* Maintain bid/ask levels
* Generate top market depth
* Calculate mid-price

Main Engine:

```rust
OrderBookManager
```

---

# Data Flow

```text
Binary File
    ↓
Packet Parsing
    ↓
Message Enum Creation
    ↓
Sequential Processing
    ↓
Orderbook Updates
    ↓
Market Depth Extraction
    ↓
Python Strategy Layer
```

---

# Installation

# Rust Requirements

Install Rust:

```bash
curl https://sh.rustup.rs -sSf | sh
```

Verify:

```bash
rustc --version
cargo --version
```

---

# Python Requirements

```bash
pip install maturin
```

---

# Build Instructions

## Development Build

```bash
maturin develop
```

## Release Build

```bash
maturin develop --release
```

## Build Wheel

```bash
maturin build --release
```

---

# Python Integration

The module is exposed as:

```python
import fastreader
```

Main class:

```python
fastreader.ReadMsgFromBinary
```

---

# API Reference

# Constructor

## ReadMsgFromBinary

```python
reader = fastreader.ReadMsgFromBinary(
    path,
    token=None
)
```

Parameters:

| Parameter | Type | Description                |
| --------- | ---- | -------------------------- |
| path      | str  | Binary market-data file    |
| token     | int  | Optional instrument filter |

---

# Methods

# all_messages

Returns all messages.

```python
reader.all_messages(limit=None)
```

Example:

```python
msgs = reader.all_messages(10)

for msg in msgs:
    print(msg)
```

---

# order_messages

Returns only order messages.

```python
reader.order_messages(limit=None)
```

Example:

```python
orders = reader.order_messages(5)
```

---

# trade_messages

Returns only trade messages.

```python
reader.trade_messages(limit=None)
```

Example:

```python
trades = reader.trade_messages(5)
```

---

# next_message

Returns next sequential message.

```python
msg = reader.next_message()
```

Example:

```python
while True:
    msg = reader.next_message()

    if msg == "END":
        break

    print(msg)
```

---

# print_messages

Print sequential messages internally.

```python
reader.print_messages(limit=10)
```

---

# orderbook

Generates orderbook snapshots.

```python
reader.orderbook()
```

Output contains:

* local timestamp
* exchange timestamp
* mid price
* top 5 bid levels
* top 5 ask levels

---

# Usage Examples

# Basic Parsing

```python
import fastreader

reader = fastreader.ReadMsgFromBinary(
    "/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin"
)

msgs = reader.all_messages(5)

for msg in msgs:
    print(msg)
```

---

# Token Based Filtering

```python
import fastreader

reader = fastreader.ReadMsgFromBinary(
    "/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin",
    token=1333
)

msgs = reader.all_messages()
```

---

# Sequential Processing

```python
import fastreader

reader = fastreader.ReadMsgFromBinary(
    "/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin"
)

while True:

    msg = reader.next_message()

    if msg == "END":
        break

    print(msg)
```

---

# Generate Orderbook

```python
import fastreader

reader = fastreader.ReadMsgFromBinary(
    "/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin" , token = 1333
)

reader.orderbook()
```

---

# Orderbook Engine

The orderbook system processes:

* order inserts
* modifications
* cancellations
* trade executions

It continuously maintains:

* best bid
* best ask
* market depth
* mid-price

Example output:

```text
local_ts,
exch_ts,
mid_price,
bid1_price,bid1_qty,
ask1_price,ask1_qty,
...
```

---

# Performance Considerations

## Why Rust?

Rust provides:

* memory safety
* no garbage collection pauses
* deterministic performance
* low latency
* high throughput

## Parsing Strategy

Current implementation loads all messages into memory.

Advantages:

* fast sequential access
* simpler architecture
* efficient repeated iteration

Tradeoff:

* high memory usage for very large files

---

# Error Handling

Errors from binary parsing are converted into Python exceptions.

Example:

```rust
.map_err(|e|
    pyo3::exceptions::PyRuntimeError
        ::new_err(e.to_string())
)?;
```

Python side:

```python
try:
    reader = fastreader.ReadMsgFromBinary(
        "bad.bin"
    )
except RuntimeError as e:
    print(e)
```

---

# Example Output

## Order Message

```text
Order Message: SeqNo1, msg_len72,
Msg_Type'O', Exch_ts123456789,
local_ts123456999,
order_id1001,
Token26000,
order_Type'B',
Price25000,
Quantity100,
missed0
```

## Trade Message

```text
Trade Message: SeqNo2, msg_len64,
Msg_Type'T', Exch_ts123456790,
local_ts123457000,
order_id_buy1001,
order_id_sell2002,
Token26000,
Price25010,
Quantity50,
missed0
```

---

# Recommended Improvements

## 1. Streaming Parser

Avoid loading all messages into memory.

Recommended:

```text
Iterator-based packet parsing
```

Benefit:

* lower memory usage
* scalable for huge datasets

---

## 2. Parallel Processing

Current processing is sequential.

Potential upgrade:

* Rayon
* Tokio
* multi-threaded token partitions

---

## 3. Structured Return Types

Currently methods return formatted strings.

Recommended:

Return structured Python dictionaries:

```python
{
    "seq_no": 1,
    "price": 25000,
    "qty": 100
}
```

Benefit:

* easier pandas integration
* faster downstream analytics

---

## 4. CSV / Parquet Export

Add direct exporters.

Example:

```python
reader.to_parquet("data.parquet")
```

---

## 5. Async Streaming

Support real-time exchange feeds.

Possible integrations:

* Kafka
* Redis Streams
* WebSockets
* UDP multicast

---

## 6. Benchmarking

Add Criterion benchmarks.

Measure:

* parsing latency
* throughput
* memory footprint
* orderbook update speed

---

# Production Architecture Recommendation

```text
Exchange Feed
      ↓
Binary Capture Layer
      ↓
Rust Decoder Engine
      ↓
Shared Memory Queue
      ↓
Orderbook Engine
      ↓
Strategy Engine
      ↓
Risk Layer
      ↓
Execution Engine
```

---

# Future Enhancements

* zero-copy parsing
* SIMD optimizations
* memory mapped files
* snapshot recovery
* incremental replay
* persistent orderbook state
* exchange gateway integration
* real-time analytics
* Python dataframe support
* Arrow / Polars integration

---

# License

MIT License

---

# Summary

FastReader is a professional-grade foundation for building:

* HFT infrastructure
* market replay systems
* backtesting engines
* orderbook analytics
* market microstructure research
* execution simulators
* exchange feed handlers

The modular Rust + Python design makes it highly extensible for quantitative trading systems.

