---
phase: 06-memory-efficiency-refactoring-p0-findings-from-issue-300-with-oracle-bit-exact-regression-tests
plan: 01
type: execute
wave: 1
depends_on: []
files_modified:
  - tests/fixtures/golden_bars_btcusdt_250_2024-01-01.parquet
  - tests/test_oracle_regression.py
  - crates/opendeviationbar-core/tests/golden_snapshot_test.rs
  - crates/opendeviationbar-core/tests/fixtures/golden_bars_btcusdt_250_2024-01-01.json
autonomous: true
requirements: [MEM-01, MEM-02, MEM-03, MEM-04, MEM-05, MEM-06, MEM-07, MEM-08]

must_haves:
  truths:
    - "Golden snapshot captured BEFORE any memory refactoring code changes"
    - "Rust golden fixture covers BTCUSDT 2024-01-01 at threshold=250 with full microstructure features"
    - "Python golden fixture covers same data with identical OHLCV + trade IDs + timestamps"
    - "Both fixtures include orphan bar (day boundary reset) for Ouroboros verification"
    - "Re-running either test suite against the current codebase produces PASS"
  artifacts:
    - path: "tests/fixtures/golden_bars_btcusdt_250_2024-01-01.parquet"
      provides: "Python-side golden snapshot fixture"
      contains: "Parquet file with OHLCV + trade IDs + microstructure features"
    - path: "tests/test_oracle_regression.py"
      provides: "Python oracle regression test"
      contains: "def test_oracle_bit_exact_regression"
    - path: "crates/opendeviationbar-core/tests/golden_snapshot_test.rs"
      provides: "Rust oracle regression test"
      contains: "fn test_golden_snapshot_bit_exact"
    - path: "crates/opendeviationbar-core/tests/fixtures/golden_bars_btcusdt_250_2024-01-01.json"
      provides: "Rust-side golden snapshot fixture"
      contains: "JSON array of bar objects with OHLCV + trade IDs + microstructure"
  key_links:
    - from: "tests/test_oracle_regression.py"
      to: "tests/fixtures/golden_bars_btcusdt_250_2024-01-01.parquet"
      via: "pl.read_parquet() fixture load"
      pattern: "read_parquet.*golden_bars"
    - from: "crates/opendeviationbar-core/tests/golden_snapshot_test.rs"
      to: "tests/fixtures/BTCUSDT-aggTrades-2024-01-01.csv"
      via: "CSV fixture load for trade input"
      pattern: "BTCUSDT-aggTrades-2024-01-01"
---

<objective>
Capture oracle golden snapshots for BTCUSDT 2024-01-01 at threshold=250 BEFORE any memory refactoring changes. These fixtures serve as the bit-exact regression baseline for all subsequent refactoring plans.

Purpose: Every P0 finding refactoring (plans 02-04) must prove output identity against these golden fixtures. The fixtures must be captured from the CURRENT code to establish the oracle baseline.

Output: Rust golden fixture (JSON), Python golden fixture (Parquet), and regression test suites for both.
</objective>

<execution_context>
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/06-memory-efficiency-refactoring-p0-findings-from-issue-300-with-oracle-bit-exact-regression-tests/06-CONTEXT.md

@crates/opendeviationbar-core/src/trade.rs
@crates/opendeviationbar-core/src/arrow_export.rs
@crates/opendeviationbar-core/src/processor.rs
@crates/opendeviationbar-core/src/test_data_loader.rs
@tests/fixtures/BTCUSDT-aggTrades-2024-01-01.csv
</context>

<tasks>

<task type="auto">
  <name>Task 1: Create Rust golden snapshot test and fixture</name>
  <files>
    crates/opendeviationbar-core/tests/golden_snapshot_test.rs
    crates/opendeviationbar-core/tests/fixtures/golden_bars_btcusdt_250_2024-01-01.json
  </files>
  <read_first>
    crates/opendeviationbar-core/src/test_data_loader.rs
    crates/opendeviationbar-core/src/processor.rs
    crates/opendeviationbar-core/src/types.rs
    crates/opendeviationbar-core/tests/processor_tests.rs
    tests/fixtures/BTCUSDT-aggTrades-2024-01-01.csv
  </read_first>
  <action>
Create a Rust integration test that:

1. Loads the existing `tests/fixtures/BTCUSDT-aggTrades-2024-01-01.csv` fixture (full day, ~760K trades) using the `load_btcusdt_fixture_headerless()` pattern from `test_data_loader.rs` (8-column headerless CSV, Binance Vision format).

2. Creates an `OpenDeviationBarProcessor` at threshold_decimal_bps=250 with `include_intra_bar_features=true` and `include_inter_bar_features=true` (lookback_trade_count=500, lookback_bar_count=5).

3. Processes all trades via `process_agg_trade_records()` to produce `Vec<OpenDeviationBar>`.

4. Calls `reset_at_ouroboros()` at the end to emit the orphan bar (simulating day boundary).

5. Has TWO modes controlled by `#[cfg(feature = "generate-golden")]`:
   - **Generate mode**: Serializes all bars to JSON and writes to `tests/fixtures/golden_bars_btcusdt_250_2024-01-01.json`. Each bar serialized with: open, high, low, close (as FixedPoint i64 raw values), volume (i128 raw), open_time, close_time (microseconds), first_agg_trade_id, last_agg_trade_id, trade_count, is_orphan, data_source, all 10 intra-bar features (f64), all 16 inter-bar features (Option<f64>).
   - **Verify mode** (default): Loads the JSON fixture file, deserializes, and asserts every field of every bar matches EXACTLY (no epsilon comparison for FixedPoint fields — they are integer-exact; f64 microstructure features use `assert!((a - b).abs() < f64::EPSILON)` for NaN-safe comparison, with NaN == NaN treated as equal).

6. The test function is named `test_golden_snapshot_bit_exact` and requires the `test-utils` feature.

7. First RUN the test in generate mode to create the fixture: `cargo nextest run -p opendeviationbar-core --features test-utils,generate-golden test_golden_snapshot_bit_exact`. Then run in verify mode: `cargo nextest run -p opendeviationbar-core --features test-utils test_golden_snapshot_bit_exact`.

To implement the generate-golden feature flag, add to `crates/opendeviationbar-core/Cargo.toml`:

```toml
[features]
generate-golden = []
```

The JSON fixture structure per bar:

```json
{
  "open_raw": 4206530000000,
  "high_raw": 4207000000000,
  "low_raw": 4206000000000,
  "close_raw": 4206800000000,
  "volume_raw": "1234567890123",
  "open_time": 1705276800000000,
  "close_time": 1705276812345678,
  "first_agg_trade_id": 3245678901,
  "last_agg_trade_id": 3245679100,
  "trade_count": 200,
  "is_orphan": false,
  "ofi": 0.123,
  "vwap_close_deviation": -0.045,
  ...
}
```

Use `serde_json` for serialization/deserialization with `#[derive(Serialize, Deserialize)]` on a dedicated `GoldenBar` struct that mirrors `OpenDeviationBar` fields but uses raw integer representations for FixedPoint (avoiding any f64 round-trip for prices/volumes).
</action>
<verify>
<automated>cd /Users/terryli/eon/opendeviationbar-py && cargo nextest run -p opendeviationbar-core --features test-utils test_golden_snapshot_bit_exact 2>&1 | tail -5</automated>
</verify>
<acceptance_criteria> - `crates/opendeviationbar-core/tests/golden_snapshot_test.rs` exists and contains `fn test_golden_snapshot_bit_exact` - `crates/opendeviationbar-core/tests/fixtures/golden_bars_btcusdt_250_2024-01-01.json` exists and is non-empty - `cargo nextest run -p opendeviationbar-core --features test-utils test_golden_snapshot_bit_exact` exits 0 - The JSON fixture contains bars with `is_orphan` field (at least one orphan bar from reset_at_ouroboros) - The JSON fixture contains bars with inter-bar feature fields (lookback_trade_count, etc.)
</acceptance_criteria>
<done>Rust golden snapshot fixture generated and verification test passes. All bars including orphan bar and microstructure features are captured.</done>
</task>

<task type="auto">
  <name>Task 2: Create Python golden snapshot test and fixture</name>
  <files>
    tests/test_oracle_regression.py
    tests/fixtures/golden_bars_btcusdt_250_2024-01-01.parquet
  </files>
  <read_first>
    tests/conftest.py
    tests/test_rust_bindings.py
    python/opendeviationbar/__init__.py
    python/opendeviationbar/orchestration/helpers.py
    src/core_bindings.rs
  </read_first>
  <action>
Create a Python test that captures and verifies golden snapshots through the full Python pipeline (PyO3 boundary included):

1. **Fixture generation script** (run once, embedded in a helper function):
   - Load `tests/fixtures/BTCUSDT-aggTrades-2024-01-01.csv` using Polars `read_csv` with schema: `agg_trade_id:i64, price:f64, quantity:f64, first_trade_id:i64, last_trade_id:i64, timestamp:i64, is_buyer_maker:bool, is_best_match:bool` (headerless, 8 columns).
   - Convert to Arrow RecordBatch via `.to_arrow()`.
   - Create `PyOpenDeviationBarProcessor(threshold_decimal_bps=250)` with `enable_inter_bar_features(lookback_trade_count=500, lookback_bar_count=5)` and `enable_intra_bar_features()`.
   - Process via `processor.process_trades_arrow(batch)` to get output RecordBatch.
   - Convert output to Polars DataFrame via `pl.from_arrow()`.
   - Include the incomplete/orphan bar from `processor.get_incomplete_bar()` if present (convert dict to single-row DataFrame and concat).
   - Save to `tests/fixtures/golden_bars_btcusdt_250_2024-01-01.parquet` with ZSTD compression.

2. **Regression test** (`test_oracle_bit_exact_regression`):
   - Regenerate bars using the same pipeline as step 1.
   - Load the golden fixture via `pl.read_parquet()`.
   - Assert row counts match exactly.
   - For each column: assert values match. For i64/bool columns, assert exact equality. For f64 columns (prices, microstructure features), assert bit-exact equality via `(golden - actual).abs().max() == 0.0` (NOT approximate — these are deterministic computations on the same input). Handle NaN columns with `.fill_nan(None)` before comparison.
   - Assert at least one bar has `is_orphan == True`.
   - Assert `first_agg_trade_id` and `last_agg_trade_id` columns are present and non-zero.

3. **Test marker**: `@pytest.mark.slow` (the fixture is ~760K trades).

4. The test should be runnable with: `uv run --python 3.13 pytest tests/test_oracle_regression.py -v -x`

5. If the golden fixture does not exist yet, the test should GENERATE it on first run (with a print message "Generating golden fixture...") and then VERIFY on subsequent runs. Use a `GENERATE_GOLDEN=1` env var to force regeneration.
   </action>

  <verify>
    <automated>cd /Users/terryli/eon/opendeviationbar-py && uv run --python 3.13 pytest tests/test_oracle_regression.py -v -x 2>&1 | tail -10</automated>
  </verify>
  <acceptance_criteria>
    - `tests/test_oracle_regression.py` exists and contains `def test_oracle_bit_exact_regression`
    - `tests/fixtures/golden_bars_btcusdt_250_2024-01-01.parquet` exists and is non-empty
    - `pytest tests/test_oracle_regression.py -v -x` exits 0
    - The Parquet fixture contains columns: open, high, low, close, volume, first_agg_trade_id, last_agg_trade_id, is_orphan
    - The test asserts bit-exact equality (not approximate) for all numeric columns
  </acceptance_criteria>
  <done>Python golden snapshot fixture generated and verification test passes. Full PyO3 boundary covered. Orphan bar and microstructure features included.</done>
</task>

</tasks>

<verification>
Both golden snapshot tests pass against the CURRENT (pre-refactoring) codebase:
- `cargo nextest run -p opendeviationbar-core --features test-utils test_golden_snapshot_bit_exact` exits 0
- `pytest tests/test_oracle_regression.py -v -x` exits 0
</verification>

<success_criteria>

- Golden fixtures exist for both Rust and Python paths
- Both test suites pass, proving the oracle baseline is consistent
- Fixtures include orphan bars (Ouroboros boundary) and full microstructure features
- No code changes to production files in this plan — only test files and fixtures
  </success_criteria>

<output>
After completion, create `.planning/phases/06-memory-efficiency-refactoring-p0-findings-from-issue-300-with-oracle-bit-exact-regression-tests/06-01-SUMMARY.md`
</output>
