---
phase: 36-fan-out-pyo3-bridge
plan: "01"
type: execute
wave: 1
depends_on: []
files_modified:
  - crates/opendeviationbar-streaming/src/clickhouse_writer/guards.rs
  - crates/opendeviationbar-streaming/src/clickhouse_writer/mod.rs
  - crates/opendeviationbar-streaming/src/live_engine/trade_dispatch.rs
  - crates/opendeviationbar-streaming/src/live_engine/mod.rs
  - crates/opendeviationbar-streaming/src/live_engine/symbol_task.rs
autonomous: true
requirements: [INTEG-01, INTEG-02, INTEG-05, INTEG-07]

must_haves:
  truths:
    - "Pre-write guards reject bars with timestamps outside [1e15, 2.5e15] microsecond range"
    - "Pre-write guards reject bars with negative volume"
    - "Fan-out dispatches completed bars to optional BarSink vector alongside ring buffer"
    - "ClickHouseWriterSink errors never block ring buffer bar delivery to Python"
    - "Ouroboros mode is hardcoded to aion in ClickHouseBarRow (already done in Phase 34)"
    - "ClickHouseWriterConfig reads OPENDEVIATIONBAR_CH_HOSTS from env (already done in Phase 34)"
  artifacts:
    - path: "crates/opendeviationbar-streaming/src/clickhouse_writer/guards.rs"
      provides: "Pre-write guard validation functions"
      exports: ["validate_bar_for_write"]
    - path: "crates/opendeviationbar-streaming/src/live_engine/trade_dispatch.rs"
      provides: "Fan-out dispatch to optional sinks after ring buffer push"
      contains: "extra_sinks"
    - path: "crates/opendeviationbar-streaming/src/live_engine/mod.rs"
      provides: "LiveBarEngine with optional Vec<Box<dyn BarSink>> field"
      contains: "extra_sinks"
  key_links:
    - from: "trade_dispatch.rs"
      to: "guards.rs"
      via: "validate_bar_for_write() called before sink.on_bar()"
      pattern: "validate_bar_for_write"
    - from: "trade_dispatch.rs"
      to: "BarSink::on_bar()"
      via: "fan-out loop over extra_sinks"
      pattern: "extra_sinks.*on_bar"
    - from: "LiveBarEngine::new()"
      to: "ClickHouseWriterSink::new()"
      via: "conditional creation when clickhouse-sink feature enabled"
      pattern: "ClickHouseWriterSink"
---

<objective>
Wire ClickHouseWriterSink into the LiveBarEngine bar dispatch pipeline with pre-write guards and fault-isolated fan-out.

Purpose: Every completed bar dispatches to both the ring buffer (Python consumer) and ClickHouseWriterSink (direct CH writes), with Rust-side validation preventing invalid bars from reaching the writer.

Output: guards.rs with pre-write validation, trade_dispatch.rs with fan-out loop, LiveBarEngine holding optional sinks, symbol_task threading sinks through.
</objective>

<execution_context>
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/36-fan-out-pyo3-bridge/36-RESEARCH.md

@crates/opendeviationbar-streaming/src/engine/traits.rs
@crates/opendeviationbar-streaming/src/clickhouse_writer/sink.rs
@crates/opendeviationbar-streaming/src/clickhouse_writer/config.rs
@crates/opendeviationbar-streaming/src/live_engine/trade_dispatch.rs
@crates/opendeviationbar-streaming/src/live_engine/mod.rs
@crates/opendeviationbar-streaming/src/live_engine/symbol_task.rs
@crates/opendeviationbar-streaming/src/engine/odb_engine.rs

<interfaces>
<!-- Key types and contracts the executor needs -->

From engine/traits.rs:

```rust
pub trait BarSink: Send {
    fn on_bar(&mut self, bar: &CompletedBar) -> Result<(), SinkError>;
    fn flush(&mut self) -> Result<(), SinkError>;
    fn name(&self) -> &str;
}

pub enum SinkError {
    Recoverable(String),
    Unrecoverable(String),
}
```

From clickhouse_writer/sink.rs:

```rust
pub struct ClickHouseWriterSink { ... }
impl BarSink for ClickHouseWriterSink { ... }
impl ClickHouseWriterSink {
    pub fn new(config: ClickHouseWriterConfig) -> Self;
    pub fn metrics(&self) -> &FlushThreadMetrics;
}
```

From clickhouse_writer/config.rs:

```rust
impl ClickHouseWriterConfig {
    pub fn from_env() -> Self;  // Reads OPENDEVIATIONBAR_CH_HOSTS etc.
}
```

From live_engine/types.rs:

```rust
pub struct CompletedBar {
    pub symbol: Arc<str>,
    pub threshold_decimal_bps: u32,
    pub bar: OpenDeviationBar,
}
```

From flush_thread.rs:

```rust
pub struct FlushThreadMetrics {
    pub bars_flushed: AtomicU64,
    pub bars_failed: AtomicU64,
    pub flush_count: AtomicU64,
    pub last_flush_latency_us: AtomicU64,
    pub dead_letter_count: AtomicU64,
    pub dead_letter_replayed: AtomicU64,
}
```

</interfaces>
</context>

<tasks>

<task type="auto" tdd="true">
  <name>Task 1: Pre-write guards module</name>
  <files>crates/opendeviationbar-streaming/src/clickhouse_writer/guards.rs, crates/opendeviationbar-streaming/src/clickhouse_writer/mod.rs</files>
  <read_first>
    - crates/opendeviationbar-streaming/src/clickhouse_writer/mod.rs
    - crates/opendeviationbar-streaming/src/clickhouse_writer/row.rs (for ClickHouseBarRow::from_completed_bar)
    - python/opendeviationbar/clickhouse/bulk_operations.py (lines 42-120, Python guards to port)
    - crates/opendeviationbar-streaming/src/live_engine/trade_dispatch.rs
  </read_first>
  <behavior>
    - validate_bar_for_write(&CompletedBar) returns true for a bar with open_time=1_700_000_000_000_000 (valid us range)
    - validate_bar_for_write returns false and logs error for bar with close_time=1_700_000_000_000 (millisecond scale, below 1e15)
    - validate_bar_for_write returns false for bar with open_time=3_000_000_000_000_000 (beyond year 2049)
    - validate_bar_for_write returns true for bar with volume=0.0 (zero is valid, only negative rejected)
    - validate_bar_for_write returns false for bar where volume is negative (overflow indicator)
    - validate_bar_for_write returns true for a normal bar (all fields valid)
  </behavior>
  <action>
    Create `guards.rs` in `crates/opendeviationbar-streaming/src/clickhouse_writer/` implementing INTEG-05.

    Port from Python `_guard_close_time_us_scale()` and `_guard_volume_non_negative()` in `bulk_operations.py`:

    ```rust
    // guards.rs
    use crate::live_engine::CompletedBar;

    const MIN_US: i64 = 1_000_000_000_000_000;  // ~2001-09-09
    const MAX_US: i64 = 2_500_000_000_000_000;  // ~2049-03-22

    pub fn validate_bar_for_write(bar: &CompletedBar) -> bool {
        let b = &bar.bar;

        // Guard 1: close_time scale (USEC-03)
        if b.close_time < MIN_US || b.close_time > MAX_US {
            tracing::error!(symbol=%bar.symbol, threshold=bar.threshold_decimal_bps,
                close_time_us=b.close_time, "pre-write guard: close_time_us out of microsecond range");
            return false;
        }
        if b.open_time < MIN_US || b.open_time > MAX_US {
            tracing::error!(symbol=%bar.symbol, threshold=bar.threshold_decimal_bps,
                open_time_us=b.open_time, "pre-write guard: open_time_us out of microsecond range");
            return false;
        }

        // Guard 2: volume non-negative (Issue #88)
        let vol = b.volume.to_f64();
        if vol < 0.0 {
            tracing::error!(symbol=%bar.symbol, threshold=bar.threshold_decimal_bps,
                volume=vol, "pre-write guard: negative volume (overflow)");
            return false;
        }

        // Guard 3: bar spread invariant (Issue #112) -- LOG ONLY, never reject
        // Wide bars are expected during flash crashes with open-anchored detection

        true
    }
    ```

    Add `pub mod guards;` to `clickhouse_writer/mod.rs` (gated by the existing feature -- all of clickhouse_writer is behind `#[cfg(feature = "clickhouse-sink")]`).

    Include unit tests in guards.rs that create CompletedBar fixtures using `OpenDeviationBar::new(&trade)` with crafted timestamps and volumes, then assert validate_bar_for_write() returns the expected result.

  </action>
  <verify>
    <automated>cargo nextest run -p opendeviationbar-streaming --features clickhouse-sink -E 'test(guard)'</automated>
  </verify>
  <acceptance_criteria>
    - grep -q "pub fn validate_bar_for_write" crates/opendeviationbar-streaming/src/clickhouse_writer/guards.rs
    - grep -q "pub mod guards" crates/opendeviationbar-streaming/src/clickhouse_writer/mod.rs
    - grep -q "MIN_US" crates/opendeviationbar-streaming/src/clickhouse_writer/guards.rs
    - grep -q "volume" crates/opendeviationbar-streaming/src/clickhouse_writer/guards.rs
  </acceptance_criteria>
  <done>Pre-write guard module exists with timestamp range and volume validation, all guard tests pass</done>
</task>

<task type="auto">
  <name>Task 2: Fan-out wiring in LiveBarEngine and trade_dispatch</name>
  <files>crates/opendeviationbar-streaming/src/live_engine/trade_dispatch.rs, crates/opendeviationbar-streaming/src/live_engine/mod.rs, crates/opendeviationbar-streaming/src/live_engine/symbol_task.rs</files>
  <read_first>
    - crates/opendeviationbar-streaming/src/live_engine/trade_dispatch.rs (full file)
    - crates/opendeviationbar-streaming/src/live_engine/mod.rs (LiveBarEngine struct, new(), start(), fill_from_rest())
    - crates/opendeviationbar-streaming/src/live_engine/symbol_task.rs (full file -- understand all parameters and the trade processing loop)
    - crates/opendeviationbar-streaming/src/engine/odb_engine.rs (fan_out() pattern, lines 189-200)
    - crates/opendeviationbar-streaming/src/clickhouse_writer/sink.rs (ClickHouseWriterSink::new, on_bar takes &CompletedBar)
    - crates/opendeviationbar-streaming/src/clickhouse_writer/flush_thread.rs (FlushThreadMetrics)
    - crates/opendeviationbar-streaming/src/clickhouse_writer/config.rs (ClickHouseWriterConfig::from_env)
    - crates/opendeviationbar-streaming/Cargo.toml (feature flags)
  </read_first>
  <action>
    Wire fan-out dispatch implementing INTEG-01 (fan-out), INTEG-02 (env config), INTEG-07 (aion already done).

    **1. LiveBarEngine (mod.rs) -- add optional sinks field:**

    Add to the `LiveBarEngine` struct:
    ```rust
    #[cfg(feature = "clickhouse-sink")]
    extra_sinks: Vec<Box<dyn crate::engine::traits::BarSink>>,
    #[cfg(feature = "clickhouse-sink")]
    ch_writer_metrics: Option<Arc<crate::clickhouse_writer::flush_thread::FlushThreadMetrics>>,
    ```

    In `LiveBarEngine::new()`:
    - When `clickhouse-sink` feature is enabled, try `ClickHouseWriterConfig::from_env()`. If `OPENDEVIATIONBAR_CH_HOSTS` is set, create `ClickHouseWriterSink::new(config)`, store its `metrics()` Arc, push the sink into `extra_sinks`.
    - If env var not set, leave `extra_sinks` empty (no CH writer).
    - Log at info level: "clickhouse writer sink created" or "clickhouse writer sink skipped (OPENDEVIATIONBAR_CH_HOSTS not set)".

    Add pub method:
    ```rust
    #[cfg(feature = "clickhouse-sink")]
    pub fn ch_writer_metrics(&self) -> Option<&Arc<crate::clickhouse_writer::flush_thread::FlushThreadMetrics>> {
        self.ch_writer_metrics.as_ref()
    }
    ```

    **2. trade_dispatch.rs -- add extra_sinks parameter:**

    Add a new parameter to `process_trade_through_processors()`:
    ```rust
    #[cfg(feature = "clickhouse-sink")]
    extra_sinks: &mut [Box<dyn crate::engine::traits::BarSink>],
    ```

    After each `bar_buffer.push(completed)` call (both the normal bar emission at line ~80-96 AND the midnight orphan emission at line ~43-56), add the fan-out dispatch. Call `validate_bar_for_write(&completed)` first -- only dispatch to sinks if the bar passes guards:

    ```rust
    #[cfg(feature = "clickhouse-sink")]
    if crate::clickhouse_writer::guards::validate_bar_for_write(&completed) {
        for sink in extra_sinks.iter_mut() {
            match sink.on_bar(&completed) {
                Ok(()) => {}
                Err(crate::engine::traits::SinkError::Recoverable(msg)) => {
                    tracing::warn!(sink = sink.name(), %msg, %symbol, threshold = *threshold,
                        "sink recoverable error");
                }
                Err(crate::engine::traits::SinkError::Unrecoverable(msg)) => {
                    tracing::error!(sink = sink.name(), %msg, %symbol, threshold = *threshold,
                        "sink unrecoverable error");
                }
            }
        }
    }
    ```

    CRITICAL: The ring buffer push MUST happen BEFORE the sink dispatch. Python must always get bars regardless of sink errors.

    **3. symbol_task.rs -- thread sinks through:**

    Add `extra_sinks` parameter to `symbol_task()` function signature:
    ```rust
    #[cfg(feature = "clickhouse-sink")]
    mut extra_sinks: Vec<Box<dyn crate::engine::traits::BarSink>>,
    ```

    Pass `&mut extra_sinks` to every `process_trade_through_processors()` call in the symbol_task loop body. There are multiple call sites in symbol_task (normal trade processing, puck gap-fill trades, etc.) -- all must pass the sinks.

    **4. LiveBarEngine::start() -- pass sinks to symbol_task:**

    In `start()`, for each symbol, take ownership of the sinks from `self.extra_sinks` for the FIRST symbol only? No -- since `BarSink` requires `&mut self`, we need ONE sink instance shared across all symbol tasks. But the sink's `on_bar()` does `try_send()` to a channel, which is threadsafe.

    Better approach: Since there's only ONE `ClickHouseWriterSink` with a shared channel to a single flush thread, clone the sender. But `BarSink` trait uses `&mut self`... The cleanest solution: wrap sinks in `Arc<Mutex<Vec<Box<dyn BarSink>>>>` and pass to each symbol_task. Inside the synchronous `process_trade_through_processors()`, lock the mutex, iterate sinks, unlock. The lock is held only for the `try_send()` duration (nanoseconds).

    Actually, re-read the research more carefully. The `process_trade_through_processors()` function is synchronous (not async), called within each symbol_task's loop body. Each symbol_task runs as a separate tokio task. We need the sinks accessible from multiple tasks.

    REVISED APPROACH: Create the `ClickHouseWriterSink` in `new()`, wrap it in `Arc<std::sync::Mutex<...>>`. In `start()`, clone the Arc for each symbol_task. In `process_trade_through_processors`, accept `sink_mutex: &Arc<Mutex<Vec<Box<dyn BarSink>>>>` instead of `&mut [...]`. Lock, iterate, unlock. The Mutex contention is negligible -- `try_send` is O(1).

    Alternatively, since `std::sync::mpsc::SyncSender` is `Clone`, we could create a wrapper that clones the sender per-task. But this breaks the BarSink abstraction.

    Go with the `Arc<Mutex<Vec<Box<dyn BarSink>>>>` approach:

    In LiveBarEngine struct:
    ```rust
    #[cfg(feature = "clickhouse-sink")]
    extra_sinks: Arc<std::sync::Mutex<Vec<Box<dyn crate::engine::traits::BarSink>>>>,
    ```

    In trade_dispatch.rs, change the extra_sinks parameter type:
    ```rust
    #[cfg(feature = "clickhouse-sink")]
    extra_sinks: &std::sync::Arc<std::sync::Mutex<Vec<Box<dyn crate::engine::traits::BarSink>>>>,
    ```

    And in the fan-out body:
    ```rust
    #[cfg(feature = "clickhouse-sink")]
    if crate::clickhouse_writer::guards::validate_bar_for_write(&completed) {
        if let Ok(mut sinks) = extra_sinks.lock() {
            for sink in sinks.iter_mut() {
                // ... error handling as above
            }
        }
    }
    ```

    In symbol_task.rs, add the parameter and pass it through. In `start()`, clone the Arc for each spawned symbol_task.

    **5. fill_from_rest() -- also fan out bars to sinks:**

    The `fill_from_rest()` method in `mod.rs` also pushes bars to `bar_buffer` (lines ~570-600). These REST-fill bars must ALSO go through the fan-out. Add the same guard + sink dispatch after each `bar_buffer.push(completed)` in `fill_from_rest()`. Clone the `extra_sinks` Arc into the JoinSet closure.

    **6. Feature-flag hygiene:**

    When `clickhouse-sink` feature is NOT enabled:
    - `process_trade_through_processors` does not have the `extra_sinks` parameter
    - `symbol_task` does not have the `extra_sinks` parameter
    - `LiveBarEngine` does not have the `extra_sinks` or `ch_writer_metrics` fields
    - All call sites use `#[cfg(feature = "clickhouse-sink")]` for the additional parameter

    Use a pattern like:
    ```rust
    // In start(), when spawning symbol_task:
    #[cfg(feature = "clickhouse-sink")]
    let extra_sinks_clone = self.extra_sinks.clone();

    tokio::spawn(symbol_task(
        symbol,
        processors,
        bar_buffer,
        // ... existing params ...
        #[cfg(feature = "clickhouse-sink")]
        extra_sinks_clone,
    ));
    ```

    This requires `symbol_task` to accept the parameter conditionally. Rust supports `#[cfg]` on function parameters? No -- Rust does NOT support `#[cfg]` on individual function parameters. Instead, always pass the sinks (when feature enabled) or use a separate function signature via cfg. The cleanest pattern: always include the parameter when `clickhouse-sink` is enabled, with `_extra_sinks: ()` when disabled? No, that's clunky.

    Best approach: When `clickhouse-sink` is enabled, add the parameter. When disabled, don't. The call sites in `start()` and `fill_from_rest()` use `#[cfg]` blocks to construct the appropriate call:

    ```rust
    #[cfg(feature = "clickhouse-sink")]
    {
        tokio::spawn(symbol_task(
            /* all existing params */,
            extra_sinks_clone,
        ));
    }
    #[cfg(not(feature = "clickhouse-sink"))]
    {
        tokio::spawn(symbol_task(
            /* all existing params -- no extra_sinks */,
        ));
    }
    ```

    This is verbose but correct. Apply same pattern in `fill_from_rest()` for the `process_trade_through_processors` calls.

    **Tests:**

    Add a test in `mod.rs` tests section:
    ```rust
    #[cfg(feature = "clickhouse-sink")]
    #[test]
    fn test_fan_out_with_noop_sink() {
        // Create a bar, wrap a NoopSink in the Arc<Mutex<Vec<...>>>, call
        // process_trade_through_processors with enough trades to emit a bar,
        // verify the sink received on_bar calls.
        // Use a CountingSink that increments an AtomicU64 on on_bar().
    }
    ```

    Verify the `cargo build` without `clickhouse-sink` feature still compiles cleanly:
    ```bash
    cargo build -p opendeviationbar-streaming --features binance-integration --no-default-features
    ```

  </action>
  <verify>
    <automated>cargo nextest run -p opendeviationbar-streaming --features clickhouse-sink -E 'test(fan_out) or test(process_trade)' && cargo build -p opendeviationbar-streaming --features binance-integration --no-default-features</automated>
  </verify>
  <acceptance_criteria>
    - grep -q "extra_sinks" crates/opendeviationbar-streaming/src/live_engine/trade_dispatch.rs
    - grep -q "extra_sinks" crates/opendeviationbar-streaming/src/live_engine/mod.rs
    - grep -q "extra_sinks" crates/opendeviationbar-streaming/src/live_engine/symbol_task.rs
    - grep -q "validate_bar_for_write" crates/opendeviationbar-streaming/src/live_engine/trade_dispatch.rs
    - grep -q "ClickHouseWriterSink" crates/opendeviationbar-streaming/src/live_engine/mod.rs
    - grep -q "ch_writer_metrics" crates/opendeviationbar-streaming/src/live_engine/mod.rs
  </acceptance_criteria>
  <done>Fan-out wiring complete: bars dispatch to both ring buffer AND optional BarSink vector. Pre-write guards run before sink dispatch. Feature-flag hygiene verified -- builds cleanly with and without clickhouse-sink.</done>
</task>

</tasks>

<verification>
- `cargo nextest run -p opendeviationbar-streaming --features clickhouse-sink` -- all tests pass
- `cargo build -p opendeviationbar-streaming --features binance-integration --no-default-features` -- compiles without clickhouse-sink
- `cargo clippy -p opendeviationbar-streaming --features clickhouse-sink -- -D warnings` -- no warnings
</verification>

<success_criteria>

- Pre-write guards reject invalid timestamps and negative volumes (INTEG-05)
- Fan-out loop dispatches bars to extra sinks after ring buffer push (INTEG-01)
- ClickHouseWriterSink conditionally created from env vars (INTEG-02)
- Ring buffer push always happens first -- sink errors never block Python (INTEG-01)
- Ouroboros mode hardcoded to "aion" in row.rs (INTEG-07, already done in Phase 34)
- Feature-flag compiles cleanly both with and without clickhouse-sink
  </success_criteria>

<output>
After completion, create `.planning/phases/36-fan-out-pyo3-bridge/36-01-SUMMARY.md`
</output>
