---
phase: "23"
plan: "01"
type: implementation
autonomous: true
wave: 1
depends_on: []
requirements: [ARROW-01, ARROW-02, ARROW-03]
---

# Phase 23 Plan 01: Arrow FFI drain_bars

## Objective

Replace dict-based `drain_bars()` with Arrow RecordBatch FFI path (`drain_bars_arrow()`), eliminating ~200K Python dict allocations during sync_flush and enabling Polars zero-copy grouping.

## Context

- @crates/opendeviationbar-streaming/src/stream_manager.rs (current drain_bars)
- @src/stream_manager_bindings.rs (PyO3 drain_bars binding)
- @python/opendeviationbar/sidecar/midnight.py (\_sync_flush_rest_bars consumer)
- @crates/opendeviationbar-core/src/arrow_export.rs (existing opendeviationbar_vec_to_record_batch)

## Tasks

### Task 1: Rust Arrow export for CompletedBar vec

type="auto"

Add `completed_bars_to_record_batch()` function in `opendeviationbar-streaming` that extends the existing `opendeviationbar_vec_to_record_batch()` schema with `_symbol` (Utf8) and `_threshold` (UInt32) metadata columns. Add `drain_bars_arrow()` method on `StreamManager` that drains the ring buffer and returns a RecordBatch.

**Done criteria:**

- `completed_bars_to_record_batch()` exists in streaming crate with Arrow dep
- Returns RecordBatch with all bar fields + `_symbol` + `_threshold_decimal_bps`
- `StreamManager::drain_bars_arrow()` calls it
- Compiles with `cargo check -p opendeviationbar-streaming --features all`

### Task 2: PyO3 drain_bars_arrow binding

type="auto"

Add `drain_bars_arrow()` method on `PyStreamManager` that returns `PyRecordBatch`. Uses GIL release during the Rust drain + Arrow conversion.

**Done criteria:**

- `PyStreamManager.drain_bars_arrow()` exists and returns PyRecordBatch
- GIL released during drain + conversion
- Compiles with `cargo check -p opendeviationbar-py --features streaming,arrow-export`

### Task 3: Python sync_flush uses Arrow path

type="auto"

Update `_sync_flush_rest_bars()` in midnight.py to prefer `drain_bars_arrow()` when available. Convert RecordBatch to Polars DataFrame with zero-copy, group by `_symbol`+`_threshold_decimal_bps` using Polars filter/partition_by instead of Python dict iteration.

**Done criteria:**

- `_sync_flush_rest_bars` calls `drain_bars_arrow()` when available
- Falls back to `drain_bars()` dict path for older builds
- Polars zero-copy grouping replaces Python dict-based grouping
- Cross-midnight filter preserved for Day-mode symbols
- Stathera diagnostic preserved

## Verification

```bash
cargo check -p opendeviationbar-streaming --features all
cargo check -p opendeviationbar-py --features streaming,arrow-export
cargo nextest run -p opendeviationbar-streaming -- drain_bars_arrow
```

## Success Criteria

1. `StreamManager.drain_bars_arrow()` returns Arrow RecordBatch with all bar fields + \_symbol/\_threshold metadata
2. `_sync_flush_rest_bars` groups RecordBatch by (symbol, threshold) using Polars zero-copy filter/slice
3. No Python dict allocation in the Arrow path (zero `opendeviationbar_to_dict` calls)
