# Unified Engine Design: Hot Path + Cold Path

Extracted patterns from NautilusTrader, Barter-rs, hftbacktest, and Bytewax.
Applied to opendeviationbar-py's Rust+Python architecture.

## The Three Traits (Total: ~50 LOC Rust)

### Trait 1: Processor (from barter-rs engine/mod.rs:75)

```rust
/// Universal event processor. Everything implements this.
/// Your bar processor, your strategy, your ClickHouse writer.
pub trait Processor<Event> {
    type Output;
    fn process(&mut self, event: Event) -> Self::Output;
}
```

**Implementations you'd write:**

```rust
// Your existing Rust processor — already does this, just formalize the trait
impl Processor<AggTrade> for OpenDeviationBarProcessor {
    type Output = Option<CompletedBar>;
    fn process(&mut self, trade: AggTrade) -> Option<CompletedBar> {
        self.process_single_trade(trade)
    }
}

// Strategy — receives completed bars, emits decisions
impl Processor<CompletedBar> for MyStrategy {
    type Output = Option<Signal>;
    fn process(&mut self, bar: CompletedBar) -> Option<Signal> {
        // In-memory decision. No ClickHouse. No I/O. Microsecond.
        if bar.ofi > 0.5 { Some(Signal::Buy) } else { None }
    }
}

// ClickHouse writer — receives bars, persists (async, off hot path)
impl Processor<CompletedBar> for ClickHouseWriter {
    type Output = ();
    fn process(&mut self, bar: CompletedBar) {
        self.batch_buffer.push(bar);  // Async flush later
    }
}
```

### Trait 2: BarSource (inspired by Bytewax PartitionedInput + barter-rs MarketStream)

```rust
/// Swappable data source. Live WS and historical replay implement the same trait.
pub trait BarSource: Send {
    /// Get next completed bar. None = no more data (backtest end) or timeout.
    fn next_bar(&mut self, timeout: Duration) -> Option<CompletedBar>;

    /// Snapshot current position for recovery (Ouroboros checkpoint).
    /// Returns None if source doesn't support recovery.
    fn snapshot(&self) -> Option<SourceCheckpoint>;
}

pub struct SourceCheckpoint {
    pub last_agg_trade_id: i64,  // Stathera anchor
    pub timestamp_ms: i64,
}
```

**Implementations:**

```rust
// Live: your existing LiveBarEngine, just implement the trait
impl BarSource for LiveBarSource {
    fn next_bar(&mut self, timeout: Duration) -> Option<CompletedBar> {
        // WebSocket → Rust processor → CompletedBar (your existing code)
        self.engine.next_bar(timeout)
    }
    fn snapshot(&self) -> Option<SourceCheckpoint> {
        Some(SourceCheckpoint {
            last_agg_trade_id: self.engine.last_agg_trade_id(),
            timestamp_ms: Utc::now().timestamp_millis(),
        })
    }
}

// Backtest: replay from Parquet/ClickHouse
impl BarSource for BacktestBarSource {
    fn next_bar(&mut self, _timeout: Duration) -> Option<CompletedBar> {
        self.bars_iterator.next()  // Pre-loaded from Parquet catalog
    }
    fn snapshot(&self) -> Option<SourceCheckpoint> {
        self.current_position.clone()
    }
}

// Vision backfill: replay from ZIP archives
impl BarSource for VisionBarSource {
    fn next_bar(&mut self, _timeout: Duration) -> Option<CompletedBar> {
        // Vision ZIP → CSV → AggTrade → Rust processor → CompletedBar
        loop {
            let trade = self.vision_reader.next_trade()?;
            if let Some(bar) = self.processor.process(trade) {
                return Some(bar);
            }
        }
    }
    fn snapshot(&self) -> Option<SourceCheckpoint> {
        Some(SourceCheckpoint {
            last_agg_trade_id: self.processor.last_agg_trade_id(),
            timestamp_ms: self.current_date_ms,
        })
    }
}
```

### Trait 3: EngineClock (from barter-rs engine/clock.rs:14)

```rust
/// Time source. Live = wall clock. Backtest = event time.
pub trait EngineClock: Send {
    fn time(&self) -> DateTime<Utc>;
}

pub struct LiveClock;
impl EngineClock for LiveClock {
    fn time(&self) -> DateTime<Utc> { Utc::now() }
}

pub struct HistoricalClock {
    last_event_time: DateTime<Utc>,
}
impl EngineClock for HistoricalClock {
    fn time(&self) -> DateTime<Utc> { self.last_event_time }
}
```

## The Engine (~30 LOC)

```rust
/// Unified engine. Same code runs live and backtest.
pub struct OdbEngine<S: BarSource, C: EngineClock> {
    pub source: S,
    pub clock: C,
    consumers: Vec<Box<dyn Processor<CompletedBar, Output = ()> + Send>>,
}

impl<S: BarSource, C: EngineClock> OdbEngine<S, C> {
    pub fn add_consumer<P>(&mut self, consumer: P)
    where P: Processor<CompletedBar, Output = ()> + Send + 'static {
        self.consumers.push(Box::new(consumer));
    }

    /// Main loop. Runs identically in live and backtest.
    pub fn run(&mut self) {
        while let Some(bar) = self.source.next_bar(Duration::from_secs(5)) {
            // Fan-out to ALL consumers (strategy + persistence + monitoring)
            for consumer in &mut self.consumers {
                consumer.process(bar.clone());
            }
        }
    }
}
```

## Usage: Write Once, Run Both Ways

```rust
// === LIVE MODE ===
let source = LiveBarSource::new(binance_ws_config);
let clock = LiveClock;
let mut engine = OdbEngine { source, clock, consumers: vec![] };

engine.add_consumer(MyStrategy::new());           // Hot path: in-memory decision
engine.add_consumer(ClickHouseWriter::new(ch));    // Cold path: async persistence
engine.add_consumer(SseBroadcaster::new(port));    // Cold path: SSE push
engine.run();

// === BACKTEST MODE ===
let source = BacktestBarSource::from_parquet("BTCUSDT", "2024-01-01", "2024-12-31");
let clock = HistoricalClock::new(start_time);
let mut engine = OdbEngine { source, clock, consumers: vec![] };

engine.add_consumer(MyStrategy::new());            // SAME strategy code
engine.add_consumer(BacktestRecorder::new());      // Record signals for analysis
engine.run();
```

## What This Replaces

| Current (hand-rolled)             | Unified (trait-based)                    |
| --------------------------------- | ---------------------------------------- |
| `sidecar.py` main loop (1500 LOC) | `OdbEngine::run()` (~10 LOC)             |
| `populate_cache_resumable()`      | `VisionBarSource` impl                   |
| `StreamManager.next_bar()`        | `LiveBarSource` impl                     |
| Separate live/backtest code paths | One `engine.run()` call                  |
| ClickHouse in hot path            | `ClickHouseWriter` as cold-path consumer |

## Where Each Pattern Came From

| Pattern                         | Source Framework | File Reference                                |
| ------------------------------- | ---------------- | --------------------------------------------- |
| `Processor<Event>` trait        | barter-rs        | `barter/src/engine/mod.rs:75`                 |
| `EngineClock` trait             | barter-rs        | `barter/src/engine/clock.rs:14`               |
| `LiveClock` / `HistoricalClock` | barter-rs        | `barter/src/engine/clock.rs:27-83`            |
| `BarSource` (swappable input)   | Bytewax          | `PartitionedInput` + `StatefulSource` pattern |
| `snapshot()` for recovery       | Bytewax          | `StatefulSource.snapshot()` → SQLite          |
| MessageBus fan-out              | NautilusTrader   | `data/engine.pyx:2254` topic-based pub        |
| Consumer fan-out pattern        | NautilusTrader   | `msgbus.publish_c()` to all subscribers       |
| Dual timestamp (exch/local)     | hftbacktest      | `(exch_ts, local_ts)` feed latency model      |
| `agg_trade_id` preservation     | barter-rs        | `BinanceTrade.id: u64` through pipeline       |

## Implementation Priority

1. **Define the 3 traits** in `opendeviationbar-streaming` (~50 LOC)
2. **Implement `LiveBarSource`** wrapping existing `LiveBarEngine` (~20 LOC)
3. **Implement `BacktestBarSource`** reading from ClickHouse/Parquet (~30 LOC)
4. **Build `OdbEngine`** with consumer fan-out (~30 LOC)
5. **Move strategy logic** from Python to Rust `Processor<CompletedBar>` impl
6. **Demote ClickHouse** to async consumer (off hot path)

Total new Rust code: ~150 LOC. No external framework dependency.
