# Phase 15: Oracle Validation - Research

**Researched:** 2026-03-27
**Domain:** Bit-exact oracle testing for Aion mode correctness
**Confidence:** HIGH

## Summary

Phase 15 creates Python-level oracle tests that prove Aion mode (continuous, no midnight reset) produces correct bars by comparing against the well-tested Day mode behavior. The three requirements (VAL-01, VAL-02, VAL-03) test single-day equivalence, multi-day continuity, and gap flag lifecycle respectively.

All the machinery is already in place: the Rust `OpenDeviationBarProcessor` with `process_trades_streaming()` and `reset_at_ouroboros()` is exposed via PyO3, gap fields (`has_gap`, `gap_trade_count`, `max_gap_duration_us`, `is_exchange_gap`) are wired through, and `maybe_reset_at_midnight()` in the streaming crate already conditionally skips reset for `OuroborosMode::Aion`. The key insight is that these oracle tests operate at the **batch processor level** (not the streaming engine level), so they feed synthetic trade sequences through `process_trades_streaming()` and compare Day-mode-with-reset vs Aion-mode-without-reset outputs.

**Primary recommendation:** Write pure Python tests using the existing `PyOpenDeviationBarProcessor` with synthetic trade generators. No new Rust code needed. Tests use `process_trades_streaming()` for trade-by-trade control and `reset_at_ouroboros()` to simulate Day mode boundaries, comparing against Aion mode (same trades, no reset).

<user_constraints>

## User Constraints (from CONTEXT.md)

### Locked Decisions

None -- auto-generated context for infrastructure/testing phase.

### Claude's Discretion

All implementation choices are at Claude's discretion -- pure testing/validation phase. Use ROADMAP phase goal, success criteria, and codebase conventions to guide decisions.

### Deferred Ideas (OUT OF SCOPE)

None -- testing phase stays within phase scope.
</user_constraints>

<phase_requirements>

## Phase Requirements

| ID     | Description                                                                                                                              | Research Support                                                                                                                                                                                                                                             |
| ------ | ---------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| VAL-01 | Oracle bit-exact test: Aion mode produces identical bars as Day mode within a single UTC day (excluding orphan bar at midnight boundary) | Day-mode processor produces bars + orphan at reset; Aion processor produces same bars minus orphan. Compare all fields except is_orphan. Uses `process_trades_streaming()` + `reset_at_ouroboros()` for Day, and `process_trades_streaming()` only for Aion. |
| VAL-02 | Multi-day oracle: Aion bars spanning UTC midnight have continuous trade IDs, no orphan, and correct microstructure features              | Feed trades spanning midnight boundary. Day mode: reset produces orphan + new bar. Aion mode: single bar spans boundary. Verify TID continuity (Stathera), no is_orphan, and VWAP/tick count correct by independent computation.                             |
| VAL-03 | Gap-flag oracle: bars with `has_gap=true` correctly identified during construction and `has_gap=false` after repair recompute            | Feed trades with TID gap -> verify has_gap=true, gap_trade_count>0, max_gap_duration_us>0. Then feed complete trades (no gap) -> verify has_gap=false.                                                                                                       |

</phase_requirements>

## Standard Stack

### Core

| Library                                             | Version         | Purpose                 | Why Standard                         |
| --------------------------------------------------- | --------------- | ----------------------- | ------------------------------------ |
| pytest                                              | 8.x (installed) | Test framework          | Already configured in pyproject.toml |
| opendeviationbar.\_core.PyOpenDeviationBarProcessor | current         | Rust processor via PyO3 | The system under test; already built |

### Supporting

| Library             | Version | Purpose                                             | When to Use                          |
| ------------------- | ------- | --------------------------------------------------- | ------------------------------------ |
| conftest.py helpers | n/a     | `make_test_trade()`, `assert_stathera_continuity()` | All tests use these shared factories |

No new dependencies needed. This phase is pure test code.

## Architecture Patterns

### Recommended Test Structure

```
tests/
└── test_aion_oracle.py    # All 3 VAL requirements in one file
```

One file is sufficient -- all three requirements share the same trade generator infrastructure and test a single conceptual domain (Aion correctness).

### Pattern 1: Day vs Aion Oracle Comparison

**What:** Feed identical trades to two processors; Day-mode gets `reset_at_ouroboros()` at midnight, Aion does not. Compare completed bars.
**When to use:** VAL-01 (single-day equivalence) and VAL-02 (multi-day continuity)
**Example:**

```python
# Source: codebase analysis of existing oracle tests
from opendeviationbar.processors.core import OpenDeviationBarProcessor

def test_single_day_equivalence():
    """VAL-01: Aion == Day within a single UTC day (minus orphan)."""
    day_proc = OpenDeviationBarProcessor(threshold_decimal_bps=250)
    aion_proc = OpenDeviationBarProcessor(threshold_decimal_bps=250)

    trades = generate_single_day_trades()  # All within one UTC day

    day_bars = day_proc.process_trades_streaming(trades)
    day_orphan = day_proc.reset_at_ouroboros()  # Day mode: emit orphan

    aion_bars = aion_proc.process_trades_streaming(trades)
    aion_incomplete = aion_proc.get_incomplete_bar()  # No reset

    # Completed bars must be identical
    assert len(day_bars) == len(aion_bars)
    for day_bar, aion_bar in zip(day_bars, aion_bars):
        assert_bars_equal(day_bar, aion_bar)
```

### Pattern 2: Midnight-Spanning Bar Verification

**What:** Feed trades that cross UTC midnight. Verify Aion produces a single continuous bar where Day would produce orphan + new bar.
**When to use:** VAL-02 (multi-day continuity)
**Example:**

```python
def test_multi_day_continuity():
    """VAL-02: Aion bars span midnight with continuous TIDs."""
    aion_proc = OpenDeviationBarProcessor(threshold_decimal_bps=250)

    # Trades: some before midnight, some after, all within breach range
    trades = generate_midnight_spanning_trades()

    aion_bars = aion_proc.process_trades_streaming(trades)
    aion_incomplete = aion_proc.get_incomplete_bar()

    # No orphan bars produced
    # If a bar spans midnight, its open_time is before midnight and
    # close_time is after midnight (or it's the incomplete bar)

    # Verify Stathera continuity (no TID gaps)
    all_bars = aion_bars + ([aion_incomplete] if aion_incomplete else [])
    assert_stathera_continuity(all_bars)

    # Verify microstructure: independently compute VWAP from trades
    # and compare against bar VWAP
```

### Pattern 3: Gap Flag Lifecycle

**What:** Feed trades with a TID gap, verify gap flags set; then feed same range with complete trades, verify flags clear.
**When to use:** VAL-03 (gap flag lifecycle)
**Example:**

```python
def test_gap_flag_lifecycle():
    """VAL-03: has_gap=true with gap, has_gap=false after repair."""
    # Phase 1: Trades with TID gap (skip IDs 105-109)
    gapped_trades = [
        make_test_trade(100, ts_base, price1),
        make_test_trade(101, ts_base + 1000, price2),
        # ...gap: IDs 102-109 missing...
        make_test_trade(110, ts_base + 10000, price3),
    ]

    proc = OpenDeviationBarProcessor(threshold_decimal_bps=250)
    bars = proc.process_trades_streaming(gapped_trades)
    incomplete = proc.get_incomplete_bar()

    # The bar containing the gap should have has_gap=true
    gapped_bar = incomplete or bars[-1]
    assert gapped_bar["has_gap"] is True
    assert gapped_bar["gap_trade_count"] > 0
    assert gapped_bar["max_gap_duration_us"] > 0

    # Phase 2: Fresh processor with complete trades (no gap)
    complete_trades = generate_complete_trades(100, 120)  # All IDs present

    clean_proc = OpenDeviationBarProcessor(threshold_decimal_bps=250)
    clean_bars = clean_proc.process_trades_streaming(complete_trades)
    clean_incomplete = clean_proc.get_incomplete_bar()

    repaired_bar = clean_incomplete or clean_bars[-1]
    assert repaired_bar["has_gap"] is False
    assert repaired_bar["gap_trade_count"] == 0
    assert repaired_bar["max_gap_duration_us"] == 0
```

### Anti-Patterns to Avoid

- **Using `process_trades()` instead of `process_trades_streaming()`:** `process_trades()` resets processor state between calls. For multi-batch/stateful tests, MUST use `process_trades_streaming()` which preserves state.
- **Testing at the streaming engine level:** The streaming engine (`LiveBarEngine`) requires async Tokio runtime, WS connections, etc. Oracle tests should use the batch processor path which is deterministic and synchronous.
- **Hardcoding expected values:** Oracle tests compare Day vs Aion output, not hardcoded values. This makes tests resilient to algorithm changes.

## Don't Hand-Roll

| Problem          | Don't Build                | Use Instead                                           | Why                                                             |
| ---------------- | -------------------------- | ----------------------------------------------------- | --------------------------------------------------------------- |
| Trade generation | Custom trade builders      | `conftest.make_test_trade()`                          | Already handles all required fields                             |
| Stathera check   | Manual TID continuity loop | `conftest.assert_stathera_continuity()`               | Existing, tested helper                                         |
| VWAP computation | Complex weighted average   | Independent `sum(price*qty)/sum(qty)` from raw trades | Oracle methodology: verify against independently computed value |

## Common Pitfalls

### Pitfall 1: Timestamp Unit Mismatch

**What goes wrong:** Trade timestamps are in milliseconds (ms) when fed through Python dict path, but Rust internally uses microseconds (us). Dict export uses `_us` suffixed keys.
**Why it happens:** The Python `process_trades()` and `process_trades_streaming()` methods accept ms timestamps and convert internally. But `get_incomplete_bar()` returns a dict with `open_time_us` and `close_time_us` keys.
**How to avoid:** Use ms timestamps in trade dicts; when reading bar dicts, use `open_time_us`/`close_time_us` for time comparisons.
**Warning signs:** Bars appearing to span impossible time ranges (1000x off).

### Pitfall 2: Midnight Boundary Calculation

**What goes wrong:** Trades at exactly midnight (00:00:00.000) belong to the new day. Off-by-one in boundary calculation.
**Why it happens:** `DAY_US = 86_400_000_000`. Midnight boundary is `trade_day = trade_timestamp_us / DAY_US`. A trade at exactly midnight belongs to the new day.
**How to avoid:** For midnight-spanning tests, use timestamps like `midnight_ms - 1000` (before) and `midnight_ms + 1000` (after). Midnight in ms for 2024-01-02: `1704153600000`.
**Warning signs:** Day-mode reset not triggering when expected.

### Pitfall 3: No Breach = No Completed Bars

**What goes wrong:** Trade prices stay within threshold, so no bars complete. Test asserts on empty list.
**Why it happens:** Open deviation bars only complete when price breaches the threshold. Tiny price movements never breach.
**How to avoid:** Use the existing `_generate_breach_trades()` pattern from `test_checkpoint_oracle.py` -- oscillating sawtooth that guarantees breaches. Or use a very small threshold (e.g., 1 dbps = 0.001%).
**Warning signs:** `len(bars) == 0` unexpectedly.

### Pitfall 4: Gap Detection Requires last_agg_trade_id > 0

**What goes wrong:** Gap detection doesn't fire on the very first trade because `last_agg_trade_id` starts at 0.
**Why it happens:** Guard in `update_with_trade()`: `if trade.agg_trade_id > expected_next && self.last_agg_trade_id > 0`. The first trade always passes without gap check.
**How to avoid:** When testing gap flags, ensure the processor has already seen at least one trade before introducing the gap. Feed 2+ trades before the gap.
**Warning signs:** `has_gap` is False despite a large TID jump from trade 1 to trade 2.

### Pitfall 5: Orphan Bar from reset_at_ouroboros

**What goes wrong:** `reset_at_ouroboros()` returns None when no bar is in progress (e.g., if the last trade completed a bar cleanly).
**Why it happens:** If the last trade triggered a breach, `defer_open=true` and there's no current bar state. Reset returns None.
**How to avoid:** When comparing Day vs Aion, account for the case where Day mode produces no orphan. Only exclude orphan from comparison when one is actually produced.
**Warning signs:** NoneType errors when trying to access orphan bar fields.

## Code Examples

### Trade Generator for Single-Day Tests

```python
# Source: analysis of conftest.py + test_checkpoint_oracle.py patterns
def generate_breach_trades_single_day(
    n: int = 200,
    threshold_dbps: int = 250,
    base_price: float = 50000.0,
    date_str: str = "2024-01-01",
) -> list[dict]:
    """Generate trades that produce multiple breaches within a single UTC day.

    Uses sawtooth price pattern: rises by step until 1.5x threshold from
    base, then reverses. Guaranteed multiple breaches.
    """
    import datetime
    base_ts_ms = int(datetime.datetime(2024, 1, 1, tzinfo=datetime.timezone.utc).timestamp() * 1000)

    threshold = base_price * (threshold_dbps / 100_000)
    step = threshold / 3  # ~3 trades per breach

    trades = []
    price = base_price
    direction = 1.0

    for i in range(n):
        price += direction * step
        if abs(price - base_price) > 1.5 * threshold:
            direction *= -1.0

        trades.append({
            "agg_trade_id": 1000 + i,
            "price": round(price, 8),
            "quantity": round(1.0 + (i % 5) * 0.5, 8),
            "first_trade_id": 2000 + i * 3,
            "last_trade_id": 2000 + i * 3 + 2,
            "timestamp": base_ts_ms + i * 1000,  # 1 trade/sec, all within day
            "is_buyer_maker": (i % 2 == 0),
        })

    return trades
```

### Trade Generator for Midnight-Spanning Tests

```python
def generate_midnight_spanning_trades(
    n_before: int = 50,
    n_after: int = 50,
    threshold_dbps: int = 250,
    base_price: float = 50000.0,
) -> list[dict]:
    """Generate trades that span UTC midnight (2024-01-01 -> 2024-01-02).

    First n_before trades are in the last minutes of Jan 1,
    last n_after trades are in the first minutes of Jan 2.
    Price movement stays WITHIN threshold so no breach occurs,
    ensuring a single bar spans the boundary.
    """
    import datetime
    midnight_ms = int(datetime.datetime(2024, 1, 2, tzinfo=datetime.timezone.utc).timestamp() * 1000)

    trades = []
    # Small price jitter -- stay within threshold to avoid completing a bar
    micro_step = base_price * (threshold_dbps / 100_000) / 20  # 5% of threshold

    for i in range(n_before):
        ts = midnight_ms - (n_before - i) * 1000  # Count up to midnight
        trades.append({
            "agg_trade_id": 1000 + i,
            "price": round(base_price + (i % 3 - 1) * micro_step, 8),
            "quantity": 1.0,
            "first_trade_id": 2000 + i * 3,
            "last_trade_id": 2000 + i * 3 + 2,
            "timestamp": ts,
            "is_buyer_maker": (i % 2 == 0),
        })

    for i in range(n_after):
        ts = midnight_ms + (i + 1) * 1000  # Count up from midnight
        idx = n_before + i
        trades.append({
            "agg_trade_id": 1000 + idx,
            "price": round(base_price + (idx % 3 - 1) * micro_step, 8),
            "quantity": 1.0,
            "first_trade_id": 2000 + idx * 3,
            "last_trade_id": 2000 + idx * 3 + 2,
            "timestamp": ts,
            "is_buyer_maker": (idx % 2 == 0),
        })

    return trades
```

### Independent VWAP Computation (Oracle Methodology)

```python
def compute_vwap_from_trades(trades: list[dict]) -> float:
    """Independently compute VWAP from raw trades.

    Oracle methodology: NEVER use the system under test to generate
    expected values. Compute from first principles.
    """
    total_turnover = sum(t["price"] * t["quantity"] for t in trades)
    total_volume = sum(t["quantity"] for t in trades)
    if total_volume == 0:
        return 0.0
    return total_turnover / total_volume
```

### Bar Equality Assertion

```python
def assert_bars_equal(bar_a: dict, bar_b: dict, skip_keys: set | None = None):
    """Assert two bar dicts are field-equal, skipping specified keys.

    For float fields, uses relative tolerance of 1e-10.
    """
    skip = skip_keys or set()
    keys_a = set(bar_a.keys()) - skip
    keys_b = set(bar_b.keys()) - skip
    assert keys_a == keys_b, f"Key mismatch: only_a={keys_a - keys_b}, only_b={keys_b - keys_a}"

    for key in keys_a:
        va, vb = bar_a[key], bar_b[key]
        if isinstance(va, float) and isinstance(vb, float):
            if va == 0.0 and vb == 0.0:
                continue
            denom = max(abs(va), abs(vb), 1e-15)
            assert abs(va - vb) / denom < 1e-10, f"Float mismatch on '{key}': {va} vs {vb}"
        else:
            assert va == vb, f"Value mismatch on '{key}': {va} vs {vb}"
```

## Validation Architecture

### Test Framework

| Property           | Value                                                         |
| ------------------ | ------------------------------------------------------------- |
| Framework          | pytest 8.x                                                    |
| Config file        | pyproject.toml [tool.pytest.ini_options]                      |
| Quick run command  | `uv run --python 3.13 pytest tests/test_aion_oracle.py -v -x` |
| Full suite command | `uv run --python 3.13 pytest tests/ -x`                       |

### Phase Requirements -> Test Map

| Req ID | Behavior                                                              | Test Type | Automated Command                                                                       | File Exists? |
| ------ | --------------------------------------------------------------------- | --------- | --------------------------------------------------------------------------------------- | ------------ |
| VAL-01 | Aion == Day within single UTC day (minus orphan)                      | unit      | `uv run --python 3.13 pytest tests/test_aion_oracle.py::test_single_day_equivalence -x` | Wave 0       |
| VAL-02 | Aion bars span midnight with continuous TIDs + correct microstructure | unit      | `uv run --python 3.13 pytest tests/test_aion_oracle.py::test_multi_day_continuity -x`   | Wave 0       |
| VAL-03 | has_gap=true with gap, has_gap=false after repair recompute           | unit      | `uv run --python 3.13 pytest tests/test_aion_oracle.py::test_gap_flag_lifecycle -x`     | Wave 0       |

### Sampling Rate

- **Per task commit:** `uv run --python 3.13 pytest tests/test_aion_oracle.py -v -x`
- **Per wave merge:** `uv run --python 3.13 pytest tests/ -x`
- **Phase gate:** Full suite green before `/gsd:verify-work`

### Wave 0 Gaps

- [ ] `tests/test_aion_oracle.py` -- covers VAL-01, VAL-02, VAL-03

## Project Constraints (from CLAUDE.md)

- **Python 3.13 only** -- all `uv run` commands must use `--python 3.13`
- **Leverage Rust** -- tests use the Rust processor via PyO3, not reimplemented Python logic
- **No ClickHouse needed** -- tests are pure unit tests with synthetic trades
- **No network I/O** -- all trade data is generated in-process
- **Tests must complete < 30s** -- conftest.py header: "all tests must complete in < 30s total"
- **No breaking PyO3 API changes** -- tests use existing `process_trades_streaming()`, `reset_at_ouroboros()`, `get_incomplete_bar()` API

## Key Technical Details

### How Day Mode vs Aion Mode Differs at the Processor Level

The `OpenDeviationBarProcessor` (Rust core) does NOT know about Ouroboros modes. It processes trades into bars purely based on threshold breaches. The Ouroboros mode is handled one level up:

- **Day mode:** The caller (streaming engine or test) manually calls `reset_at_ouroboros()` at midnight boundaries. This emits an orphan bar and resets the processor.
- **Aion mode:** The caller simply never calls `reset_at_ouroboros()`. The processor continues building bars across midnight.

**This means oracle tests can simulate both modes using the same processor class** -- the difference is purely in whether `reset_at_ouroboros()` is called at midnight.

### Gap Detection Mechanics

Gap detection fires in `update_with_trade()` (types.rs line 524-538):

- Condition: `trade.agg_trade_id > expected_next && self.last_agg_trade_id > 0`
- Sets: `has_gap = true`, increments `gap_trade_count`, tracks `max_gap_duration_us`
- `is_exchange_gap` stays false (Python sets it after REST confirms trades don't exist)

Gap flags are **per-bar** (not per-trade). Once a gap is detected within a bar, `has_gap` stays true for that bar. A fresh processor starting from complete trades will produce bars with `has_gap=false`.

### Microstructure Verification for VAL-02

VAL-02 requires verifying "correct microstructure features (VWAP, tick count)" for bars spanning midnight. The approach:

1. Compute VWAP independently: `sum(price * quantity) / sum(quantity)` from the raw trades that fall within the bar's TID range
2. Verify `agg_record_count` matches the count of trades fed to the processor for that bar
3. For the spanning bar, these metrics should reflect ALL trades (pre + post midnight), not just one side

## Open Questions

1. **Whether `process_trades_streaming()` dict output includes gap fields**
   - What we know: `opendeviationbar_to_dict()` in helpers.rs exports has_gap, gap_trade_count, max_gap_duration_us, is_exchange_gap (confirmed at lines 143-146)
   - What's unclear: Whether `process_trades_streaming()` uses the same dict path
   - Recommendation: It does -- both `process_trades()` and `process_trades_streaming()` call `opendeviationbar_to_dict()`. Confirmed by code inspection of core_bindings.rs lines 362-363.

## Sources

### Primary (HIGH confidence)

- Codebase inspection: `crates/opendeviationbar-core/src/types.rs` lines 504-539 (gap detection)
- Codebase inspection: `crates/opendeviationbar-streaming/src/live_engine/trade_dispatch.rs` lines 136-160 (Aion mode)
- Codebase inspection: `src/core_bindings.rs` lines 267-366 (PyO3 processor bindings)
- Codebase inspection: `tests/test_checkpoint_oracle.py` (oracle test patterns)
- Codebase inspection: `tests/test_oracle_regression.py` (bit-exact comparison patterns)
- Codebase inspection: `tests/conftest.py` (shared test helpers)

## Metadata

**Confidence breakdown:**

- Standard stack: HIGH - pure Python tests using existing PyO3 bindings, no new deps
- Architecture: HIGH - follows established oracle test patterns in the codebase
- Pitfalls: HIGH - verified against actual code paths and existing test failures

**Research date:** 2026-03-27
**Valid until:** 2026-04-27 (stable -- testing infrastructure rarely changes)
