Metadata-Version: 2.4
Name: OrderPulse
Version: 0.2.25
Summary: High-performance exchange feed parser and orderflow analytics engine with Rust and Python bindings
Requires-Python: >=3.9
Description-Content-Type: text/markdown

# OrderPulse / rmoney-orderbook

`rmoney-orderbook` is a Rust-powered Python extension module for reading exchange binary order/trade messages and building an orderbook snapshot from them.

The library is written in Rust for speed and exposed to Python using PyO3. Python users can load raw binary market-data files, process order/trade packets, apply filters, and get bid/ask snapshots as normal Python dictionaries.

---

## Purpose

This package is designed for high-performance market-data processing.

It helps convert raw exchange binary files into useful orderbook information.

```text
Raw binary file
        ↓
Rust binary parser
        ↓
Order / Trade messages
        ↓
OrderbookBuilder
        ↓
OrderBookManager
        ↓
Python snapshot output
```

The Python user does not need to manually decode the binary file.

---

## Main Features

- Read order/trade binary files.
- Cache all parsed messages in memory.
- Process messages into an orderbook.
- Apply message-type filters.
- Build orderbook from cached messages or stream-like loader.
- Return top bid/ask levels.
- Return best bid, best ask, spread, and mid price.
- Expose Rust performance through a clean Python API.

---

## Installation

After publishing to PyPI:

```bash
pip install rmoney-orderbook
```

For local development using `maturin`:

```bash
maturin develop --release
```

Build wheel:

```bash
maturin build --release
pip install target/wheels/*.whl
```

---

## Python Import

```python
from rmoney_orderbook import MessageCacheReader, StreamingBinaryLoader, OrderbookBuilder
```

---

# Architecture Overview

The library contains three Python-facing classes:

```text
MessageCacheReader
StreamingBinaryLoader
OrderbookBuilder
```

Internally, the Rust project is organized like this:

```rust
mod orderbook;
mod orderbook_processing;
mod read_trd_ord_only;
mod structure;
mod tsc;
```

## Internal Module Purpose

| Module | Purpose |
|---|---|
| `structure` | Defines binary packet structures such as `Message`, `OrderPacket`, `TradePacket`, and `PeekStructure`. |
| `read_trd_ord_only` | Reads binary order/trade files and converts them into Rust `Message` values. |
| `orderbook` | Contains the orderbook engine such as `OrderBookManager`, `PriceLevel`, and `Side`. |
| `orderbook_processing` | Contains helper processing logic. |
| `tsc` | Contains timing/cycle utility logic. |
| `lib.rs` | Exposes Rust classes and methods to Python using PyO3. |

---

# Class 1: `MessageCacheReader`

## Purpose

`MessageCacheReader` is the RAM-based reader.

It loads the full binary file into memory and stores all parsed messages.

This is useful when:

- the file fits in RAM,
- you want to reuse the same data multiple times,
- you want fast repeated backtesting,
- you want to inspect parsed messages,
- you want summary information about the loaded file.

---

## Internal Rust Structure

```rust
#[pyclass]
pub struct MessageCacheReader {
    file_path: Option<String>,
    messages: Arc<Vec<Message>>,
}
```

## Field Explanation

| Field | Type | Purpose |
|---|---|---|
| `file_path` | `Option<String>` | Stores the source file path after loading. |
| `messages` | `Arc<Vec<Message>>` | Stores all parsed messages in memory. |

`Arc<Vec<Message>>` is used so the vector can be safely shared without copying the full message list.

---

## Constructor

```python
reader = MessageCacheReader()
```

### Purpose

Creates an empty cache reader.

### Initial State

```text
file_path = None
messages = []
```

### Expected Output

```python
<MessageCacheReader object>
```

---

## `load_to_cache(file_path)`

### Purpose

Loads the full binary file into memory.

### Python Example

```python
from rmoney_orderbook import MessageCacheReader

reader = MessageCacheReader()

count = reader.load_to_cache("/data/market_feed.bin")

print(count)
```

### Expected Output

```text
1250000
```

This means 1,250,000 messages were loaded successfully.

---

## What Happens Internally?

```text
Python passes file path
        ↓
Rust calls binary reader
        ↓
Binary packets are converted into Message values
        ↓
Messages are stored in memory
        ↓
Message count is returned to Python
```

---

## `get_all_messages()`

### Purpose

Returns all cached messages as readable strings.

This is mainly useful for debugging and validating whether the binary file is being decoded correctly.

### Python Example

```python
reader = MessageCacheReader()
reader.load_to_cache("/data/market_feed.bin")

messages = reader.get_all_messages()

print(messages[0])
print(messages[1])
```

### Expected Output

```text
Order Message: SeqNo 1001, MsgLen 38, MsgType 'N', ExchTs 1718000000000000000, LocalTs 1718000000000100000, OrderId 987654321, Token 26000, Side 'B', Price 2255000, Quantity 75, Missed 0
Trade Message: SeqNo 1002, MsgLen 45, MsgType 'T', ExchTs 1718000000000200000, LocalTs 1718000000000300000, BuyOrderId 987654321, SellOrderId 987654322, Token 26000, Price 2255050, Quantity 25, Missed 0
```

---

## `get_cache_summary()`

### Purpose

Returns metadata about the loaded file.

### Python Example

```python
reader = MessageCacheReader()
reader.load_to_cache("/data/market_feed.bin")

summary = reader.get_cache_summary()

print(summary)
```

### Expected Output

```python
{
    "file_source": "/data/market_feed.bin",
    "total_messages": 1250000,
    "total_orders": 1000000,
    "total_trades": 250000,
    "memory_usage_bytes": 90000000
}
```

### Output Explanation

| Key | Meaning |
|---|---|
| `file_source` | File path loaded into cache. |
| `total_messages` | Total number of parsed messages. |
| `total_orders` | Number of order messages. |
| `total_trades` | Number of trade messages. |
| `memory_usage_bytes` | Approximate memory used by parsed messages. |

---

# Class 2: `StreamingBinaryLoader`

## Purpose

`StreamingBinaryLoader` is a stream-style loader.

It is designed to process messages sequentially instead of exposing the full message list to Python.

In the current implementation, it stores messages internally and advances using an index. From the Python user’s perspective, it behaves like a source that can be consumed by `OrderbookBuilder`.

---

## Internal Rust Structure

```rust
#[pyclass]
pub struct StreamingBinaryLoader {
    file_path: Option<String>,
    messages: Vec<Message>,
    index: usize,
}
```

## Field Explanation

| Field | Type | Purpose |
|---|---|---|
| `file_path` | `Option<String>` | Stores opened file path. |
| `messages` | `Vec<Message>` | Stores parsed messages internally. |
| `index` | `usize` | Tracks current read position. |

---

## Constructor

```python
loader = StreamingBinaryLoader()
```

### Purpose

Creates an empty streaming loader.

### Initial State

```text
file_path = None
messages = []
index = 0
```

---

## `open_stream(file_path)`

### Purpose

Opens and prepares the file for stream-style processing.

### Python Example

```python
from rmoney_orderbook import StreamingBinaryLoader

loader = StreamingBinaryLoader()

count = loader.open_stream("/data/market_feed.bin")

print(count)
```

### Expected Output

```text
1250000
```

This means the file was opened and messages are ready for sequential processing.

---

## Internal Logic

```text
File path is received
        ↓
Binary reader parses messages
        ↓
Messages are stored inside loader
        ↓
index is reset to 0
        ↓
Message count is returned
```

---

## Internal `get_next_message()`

This method is used internally by `OrderbookBuilder`.

It is not exposed directly to Python.

### Purpose

Returns the next message and moves the internal pointer forward.

### Logic

```text
Read message at current index
        ↓
Increment index
        ↓
Return message
```

If no messages are left, it returns `None`.

---

# Class 3: `OrderbookBuilder`

## Purpose

`OrderbookBuilder` is the main engine exposed to Python.

It consumes parsed messages from `MessageCacheReader` or `StreamingBinaryLoader` and updates the internal orderbook state.

---

## Internal Rust Structure

```rust
#[pyclass]
pub struct OrderbookBuilder {
    manager: OrderBookManager,
    allowed_message_types: Option<Vec<u8>>,
}
```

## Field Explanation

| Field | Type | Purpose |
|---|---|---|
| `manager` | `OrderBookManager` | Internal bid/ask orderbook engine. |
| `allowed_message_types` | `Option<Vec<u8>>` | Optional message-type filter. |

---

## Constructor

```python
builder = OrderbookBuilder()
```

### Purpose

Creates a new empty orderbook builder.

### Initial State

```text
manager = empty OrderBookManager
allowed_message_types = None
```

When `allowed_message_types` is `None`, all messages are processed.

---

## `apply_filter(logic_criteria=None)`

### Purpose

Filters which message types should be processed.

For example, if you only want to process add/modify/delete order messages, you can pass only those message type codes.

### Python Example

```python
builder = OrderbookBuilder()

builder.apply_filter(["N", "M", "X"])
```

### Meaning

```text
Only process message types:
N
M
X
```

Messages with other types will be skipped.

---

## Why Strings Are Converted to Bytes

Python users pass strings:

```python
["N", "M", "X"]
```

But binary packets store message type as bytes:

```rust
b'N'
b'M'
b'X'
```

So internally, Rust converts:

```text
"N" -> b'N'
"M" -> b'M'
"X" -> b'X'
```

---

## If No Filter Is Applied

```python
builder = OrderbookBuilder()
```

All messages are processed.

Equivalent internal state:

```rust
allowed_message_types = None
```

---

## `build_from_list(reader)`

### Purpose

Builds the orderbook from a `MessageCacheReader`.

This is the RAM-based workflow.

### Python Example

```python
from rmoney_orderbook import MessageCacheReader, OrderbookBuilder

reader = MessageCacheReader()
reader.load_to_cache("/data/market_feed.bin")

builder = OrderbookBuilder()

processed = builder.build_from_list(reader)

print(processed)
```

### Expected Output

```text
1250000
```

This means 1,250,000 messages were processed.

---

## Internal Flow

```text
Loop through cached messages
        ↓
Check if message is Order or Trade
        ↓
Apply filter if filter exists
        ↓
Call process_order_message() or process_trade_message()
        ↓
Update orderbook state
        ↓
Return processed count
```

---

## `build_from_source(source, limit=None)`

### Purpose

Builds the orderbook from either:

```python
MessageCacheReader
```

or:

```python
StreamingBinaryLoader
```

This method is flexible because Python can pass either source type.

---

## Example with `MessageCacheReader`

```python
reader = MessageCacheReader()
reader.load_to_cache("/data/market_feed.bin")

builder = OrderbookBuilder()

processed = builder.build_from_source(reader)

print(processed)
```

### Expected Output

```text
1250000
```

---

## Example with `StreamingBinaryLoader`

```python
loader = StreamingBinaryLoader()
loader.open_stream("/data/market_feed.bin")

builder = OrderbookBuilder()

processed = builder.build_from_source(loader, 100000)

print(processed)
```

### Expected Output

```text
100000
```

This means only the first 100,000 messages were processed because a limit was provided.

---

## Why `limit` Is Useful

`limit` is useful for:

- debugging,
- testing,
- partial processing,
- avoiding long runs on huge files,
- checking first few messages before full processing.

Example:

```python
builder.build_from_source(loader, 5000)
```

This processes only 5,000 messages.

---

## Wrong Source Type Error

If a wrong object is passed:

```python
builder.build_from_source("wrong")
```

Expected error:

```text
TypeError: build_from_source expects MessageCacheReader or StreamingBinaryLoader
```

---

# `get_snapshot(token, levels=None)`

## Purpose

Returns the current orderbook snapshot for a specific token.

### Python Example

```python
snapshot = builder.get_snapshot(26000, 5)

print(snapshot)
```

### Expected Output

```python
{
    "token": 26000,
    "found": True,
    "mid_price": 2255025,
    "best_bid": (2255000, 150),
    "best_ask": (2255050, 200),
    "spread": 50,
    "bids": [
        (2255000, 150),
        (2254950, 100),
        (2254900, 75),
        (2254850, 50),
        (2254800, 25)
    ],
    "asks": [
        (2255050, 200),
        (2255100, 125),
        (2255150, 80),
        (2255200, 40),
        (2255250, 20)
    ]
}
```

---

## Output Explanation

| Key | Meaning |
|---|---|
| `token` | Instrument token requested. |
| `found` | Whether orderbook data exists for this token. |
| `mid_price` | Mid price returned by the orderbook manager. |
| `best_bid` | Top bid level as `(price, quantity)`. |
| `best_ask` | Top ask level as `(price, quantity)`. |
| `spread` | `best_ask_price - best_bid_price`. |
| `bids` | Top bid levels. |
| `asks` | Top ask levels. |

---

## If Token Is Not Found

```python
snapshot = builder.get_snapshot(99999, 5)

print(snapshot)
```

### Expected Output

```python
{
    "token": 99999,
    "found": False,
    "mid_price": 0,
    "best_bid": None,
    "best_ask": None,
    "spread": None,
    "bids": [],
    "asks": []
}
```

---

# Complete Workflow 1: Cache-Based Processing

Use this when the file can fit in RAM.

```python
from rmoney_orderbook import MessageCacheReader, OrderbookBuilder

FILE_PATH = "/data/market_feed.bin"
TOKEN = 26000

reader = MessageCacheReader()

loaded_count = reader.load_to_cache(FILE_PATH)

print("Loaded messages:", loaded_count)

summary = reader.get_cache_summary()

print("Cache summary:", summary)

builder = OrderbookBuilder()

processed_count = builder.build_from_list(reader)

print("Processed messages:", processed_count)

snapshot = builder.get_snapshot(TOKEN, 5)

print("Snapshot:", snapshot)
```

## Expected Output

```text
Loaded messages: 1250000
Cache summary: {'file_source': '/data/market_feed.bin', 'total_messages': 1250000, 'total_orders': 1000000, 'total_trades': 250000, 'memory_usage_bytes': 90000000}
Processed messages: 1250000
Snapshot: {'token': 26000, 'found': True, 'mid_price': 2255025, 'best_bid': (2255000, 150), 'best_ask': (2255050, 200), 'spread': 50, 'bids': [(2255000, 150), (2254950, 100), (2254900, 75), (2254850, 50), (2254800, 25)], 'asks': [(2255050, 200), (2255100, 125), (2255150, 80), (2255200, 40), (2255250, 20)]}
```

---

# Complete Workflow 2: Filtered Processing

Use this when you want to process only selected message types.

```python
from rmoney_orderbook import MessageCacheReader, OrderbookBuilder

reader = MessageCacheReader()
reader.load_to_cache("/data/market_feed.bin")

builder = OrderbookBuilder()

builder.apply_filter(["N", "M", "X"])

processed = builder.build_from_list(reader)

print("Processed filtered messages:", processed)

snapshot = builder.get_snapshot(26000, 5)

print(snapshot)
```

## Expected Output

```text
Processed filtered messages: 1000000
{'token': 26000, 'found': True, 'mid_price': 2255025, 'best_bid': (2255000, 150), 'best_ask': (2255050, 200), 'spread': 50, 'bids': [(2255000, 150), (2254950, 100), (2254900, 75), (2254850, 50), (2254800, 25)], 'asks': [(2255050, 200), (2255100, 125), (2255150, 80), (2255200, 40), (2255250, 20)]}
```

---

# Complete Workflow 3: Stream-Style Processing with Limit

Use this when you want to process only a limited number of messages.

```python
from rmoney_orderbook import StreamingBinaryLoader, OrderbookBuilder

loader = StreamingBinaryLoader()

loaded = loader.open_stream("/data/huge_market_feed.bin")

print("Loaded into stream source:", loaded)

builder = OrderbookBuilder()

processed = builder.build_from_source(loader, 100000)

print("Processed:", processed)

snapshot = builder.get_snapshot(26000, 5)

print(snapshot)
```

## Expected Output

```text
Loaded into stream source: 5000000
Processed: 100000
{'token': 26000, 'found': True, 'mid_price': 2255025, 'best_bid': (2255000, 150), 'best_ask': (2255050, 200), 'spread': 50, 'bids': [(2255000, 150), (2254950, 100), (2254900, 75), (2254850, 50), (2254800, 25)], 'asks': [(2255050, 200), (2255100, 125), (2255150, 80), (2255200, 40), (2255250, 20)]}
```


