---
phase: 02-midnight-preflight-verification
plan: 01
type: execute
wave: 1
depends_on: []
files_modified:
  - python/opendeviationbar/sidecar.py
  - python/opendeviationbar/clickhouse/query_operations.py
  - tests/test_midnight_preflight.py
autonomous: true
requirements: [PREFLIGHT-01, PREFLIGHT-02]

must_haves:
  truths:
    - "Sidecar can query Binance REST for the first trade at or after UTC midnight for any symbol"
    - "Sidecar can query ClickHouse for yesterday's orphan bar and its last_agg_trade_id"
    - "Verification logs INFO on perfect continuity, WARNING on mismatch or missing orphan"
  artifacts:
    - path: "python/opendeviationbar/sidecar.py"
      provides: "_get_midnight_crossover_tid() and _verify_yesterday_orphan() functions"
      contains: "def _get_midnight_crossover_tid"
    - path: "python/opendeviationbar/clickhouse/query_operations.py"
      provides: "get_yesterday_orphan_tid() query method"
      contains: "def get_yesterday_orphan_tid"
    - path: "tests/test_midnight_preflight.py"
      provides: "Unit tests for crossover query and orphan verification"
  key_links:
    - from: "python/opendeviationbar/sidecar.py"
      to: "opendeviationbar._core.fetch_aggtrades_rest"
      via: "REST call with startTime=midnight_ms, limit via end_time_ms=midnight_ms+1000"
      pattern: "fetch_aggtrades_rest.*midnight"
    - from: "python/opendeviationbar/sidecar.py"
      to: "python/opendeviationbar/clickhouse/query_operations.py"
      via: "cache.get_yesterday_orphan_tid(symbol)"
      pattern: "get_yesterday_orphan_tid"
---

<objective>
Add midnight crossover trade ID query and yesterday orphan verification functions to the sidecar startup path.

Purpose: These two functions are the foundation of midnight preflight -- querying Binance REST for the exact first trade after UTC midnight, and verifying ClickHouse's yesterday orphan bar closes at exactly crossover_tid - 1. Plan 02 will wire these into_create_engine().

Output: Two new functions in sidecar.py (\_get_midnight_crossover_tid, \_verify_yesterday_orphan), one new ClickHouse query method (get_yesterday_orphan_tid), and unit tests.
</objective>

<execution_context>
@~/.claude/get-shit-done/workflows/execute-plan.md
@~/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/02-midnight-preflight-verification/02-CONTEXT.md
@.planning/phases/01-dedup-floor-fix/01-02-SUMMARY.md

<interfaces>
<!-- Key types and contracts the executor needs -->

From python/opendeviationbar/sidecar.py (existing pattern for REST calls):

```python
# Line 447-503: _seed_trade_ids_from_clickhouse() — per-symbol ClickHouse query pattern
# Uses: from opendeviationbar.clickhouse.cache import OpenDeviationBarCache
# Uses: cache.get_latest_bar_with_trade_id(symbol, threshold)
# Uses: engine.seed_trade_id(symbol, threshold, last_tid)
# Retry pattern: stamina or manual retry with _resilience_cfg()
```

From src/binance_bindings.rs (existing PyO3 binding):

```python
# fetch_aggtrades_rest(symbol, start_time_ms, end_time_ms, market_type=None)
# Returns: list of dicts with keys: timestamp, price, quantity, agg_trade_id,
#           first_trade_id, last_trade_id, is_buyer_maker
# Note: timestamp already in ms (converted from us in Rust)
```

From python/opendeviationbar/clickhouse/query_operations.py:

```python
class QueryOperationsMixin:
    # get_latest_bar_with_trade_id(symbol, threshold, ouroboros_mode=None)
    #   -> tuple[int|None, int|None]  (close_time_ms, last_agg_trade_id)
    # Uses FINAL_READ_SETTINGS, handles CLICKHOUSE_TRANSIENT_ERRORS
```

</interfaces>
</context>

<tasks>

<task type="auto">
  <name>Task 1: Add ClickHouse query for yesterday's orphan bar trade ID</name>
  <files>python/opendeviationbar/clickhouse/query_operations.py</files>
  <read_first>
    python/opendeviationbar/clickhouse/query_operations.py
    python/opendeviationbar/clickhouse/cache.py
  </read_first>
  <action>
Add `get_yesterday_orphan_tid()` method to `QueryOperationsMixin` class in `query_operations.py`.

```python
def get_yesterday_orphan_tid(
    self,
    symbol: str,
    ouroboros_mode: str | None = None,
) -> int | None:
    """Get yesterday's orphan bar's last_agg_trade_id for a symbol.

    Queries for the orphan bar (is_orphan=1) on yesterday's UTC date.
    Returns None if no orphan bar exists (fresh deploy, gap, or non-day-mode).

    Used by midnight preflight verification (PREFLIGHT-02) to check that
    yesterday's orphan closes at exactly crossover_trade_id - 1.
    """
```

Query:

```sql
SELECT last_agg_trade_id
FROM opendeviationbar_cache.open_deviation_bars FINAL
WHERE symbol = {symbol:String}
  AND ouroboros_mode = {ouroboros:String}
  AND toDate(open_time_ms / 1000, 'UTC') = yesterday()
  AND is_orphan = 1
ORDER BY last_agg_trade_id DESC
LIMIT 1
```

Follow the same pattern as `get_latest_bar_with_trade_id()`:

- Resolve `ouroboros_mode` from config if None
- Use `FINAL_READ_SETTINGS`
- Catch `CLICKHOUSE_TRANSIENT_ERRORS`, log and return None
- Return `int | None`
  </action>
  <verify>
  <automated>cd /Users/terryli/eon/opendeviationbar-py && grep -n "def get_yesterday_orphan_tid" python/opendeviationbar/clickhouse/query_operations.py</automated>
  </verify>
  <acceptance_criteria>
  - grep "def get_yesterday_orphan_tid" python/opendeviationbar/clickhouse/query_operations.py returns a match
  - grep "is_orphan = 1" python/opendeviationbar/clickhouse/query_operations.py returns a match
  - grep "yesterday()" python/opendeviationbar/clickhouse/query_operations.py returns a match
  - Method returns int | None
  - Method handles CLICKHOUSE_TRANSIENT_ERRORS gracefully (returns None)
    </acceptance_criteria>
    <done>get_yesterday_orphan_tid() method exists in QueryOperationsMixin, queries ClickHouse for yesterday's orphan bar last_agg_trade_id, returns int|None</done>
    </task>

<task type="auto">
  <name>Task 2: Add _get_midnight_crossover_tid and _verify_yesterday_orphan to sidecar.py</name>
  <files>python/opendeviationbar/sidecar.py, tests/test_midnight_preflight.py</files>
  <read_first>
    python/opendeviationbar/sidecar.py
    python/opendeviationbar/clickhouse/query_operations.py
    src/binance_bindings.rs
  </read_first>
  <action>
Add two private functions to sidecar.py, placed after `_seed_trade_ids_from_clickhouse()` (around line 503) and before `_notify_seeding_failure()`:

**Function 1: `_get_midnight_crossover_tid(symbol: str) -> int | None`**

```python
def _get_midnight_crossover_tid(symbol: str) -> int | None:
    """Query Binance REST for the first aggregated trade at or after UTC midnight.

    PREFLIGHT-01: Uses fetch_aggtrades_rest(symbol, midnight_ms, midnight_ms + 1000)
    where midnight_ms is start-of-today UTC in milliseconds. The 1-second window
    (midnight_ms + 1000) ensures we get at most a few trades — we only need the first.

    Returns the agg_trade_id of the first trade of today, or None on failure.
    """
```

Implementation:

- Compute `midnight_ms` = `int(datetime.now(timezone.utc).replace(hour=0, minute=0, second=0, microsecond=0).timestamp() * 1000)`
- Call `from opendeviationbar._core import fetch_aggtrades_rest`
- Call `fetch_aggtrades_rest(symbol, midnight_ms, midnight_ms + 1000)`
- If result is empty list, log WARNING and return None
- Return `result[0]["agg_trade_id"]` (first trade = crossover)
- Wrap in try/except (RuntimeError, OSError), log WARNING on failure, return None
- Use `@stamina.retry(on=(RuntimeError, OSError), attempts=3, timeout=30)` decorator for resilience (import stamina at top of function or module level -- follow existing sidecar.py patterns)

**Function 2: `_verify_yesterday_orphan(symbol: str, crossover_tid: int) -> bool`**

```python
def _verify_yesterday_orphan(symbol: str, crossover_tid: int) -> bool:
    """Verify yesterday's orphan bar closes at exactly crossover_tid - 1.

    PREFLIGHT-02: Queries ClickHouse for yesterday's orphan bar and checks
    that orphan.last_agg_trade_id == crossover_tid - 1 (perfect Stathera continuity).

    Returns True if verification passes, False if mismatch or missing.
    Never raises — logs WARNING on any issue and returns False.
    """
```

Implementation:

- Call `from opendeviationbar.clickhouse.cache import OpenDeviationBarCache`
- Use `with OpenDeviationBarCache() as cache:` context manager
- Call `orphan_tid = cache.get_yesterday_orphan_tid(symbol)`
- If `orphan_tid is None`: log WARNING "no orphan bar for {symbol} yesterday — fresh deploy or gap, deferring to kintsugi", return False
- If `orphan_tid == crossover_tid - 1`: log INFO "midnight continuity verified for {symbol}: orphan_last_tid={orphan_tid}, crossover_tid={crossover_tid}", return True
- If mismatch: log WARNING "midnight continuity MISMATCH for {symbol}: orphan_last_tid={orphan_tid}, expected={crossover_tid - 1}, crossover_tid={crossover_tid} — deferring to kintsugi", return False
- Wrap outer call in try/except (Exception), log WARNING on failure, return False

**Tests: `tests/test_midnight_preflight.py`**

Create a test file with:

1. `test_get_midnight_crossover_tid_success` — mock `fetch_aggtrades_rest` to return a list with one trade dict, verify return value is the agg_trade_id
2. `test_get_midnight_crossover_tid_empty` — mock returns empty list, verify returns None
3. `test_get_midnight_crossover_tid_network_error` — mock raises RuntimeError, verify returns None (no crash)
4. `test_verify_yesterday_orphan_match` — mock `get_yesterday_orphan_tid` returns crossover_tid - 1, verify returns True
5. `test_verify_yesterday_orphan_mismatch` — mock returns wrong value, verify returns False
6. `test_verify_yesterday_orphan_no_orphan` — mock returns None, verify returns False
   </action>

  <verify>
    <automated>cd /Users/terryli/eon/opendeviationbar-py && uv run --python 3.13 pytest tests/test_midnight_preflight.py -v --no-header 2>&1 | tail -20</automated>
  </verify>
  <acceptance_criteria>
    - grep "def _get_midnight_crossover_tid" python/opendeviationbar/sidecar.py returns a match
    - grep "def _verify_yesterday_orphan" python/opendeviationbar/sidecar.py returns a match
    - grep "fetch_aggtrades_rest" python/opendeviationbar/sidecar.py returns a match in _get_midnight_crossover_tid
    - grep "get_yesterday_orphan_tid" python/opendeviationbar/sidecar.py returns a match in _verify_yesterday_orphan
    - All 6 tests in tests/test_midnight_preflight.py pass
    - _get_midnight_crossover_tid never raises (returns None on error)
    - _verify_yesterday_orphan never raises (returns False on error)
  </acceptance_criteria>
  <done>Both preflight functions exist in sidecar.py, tested with 6 unit tests covering success, mismatch, missing, and error paths. Functions never crash — they log warnings and return safe defaults.</done>
</task>

</tasks>

<verification>
- `grep -n "def get_yesterday_orphan_tid" python/opendeviationbar/clickhouse/query_operations.py` shows the ClickHouse query method
- `grep -n "def _get_midnight_crossover_tid" python/opendeviationbar/sidecar.py` shows the REST query function
- `grep -n "def _verify_yesterday_orphan" python/opendeviationbar/sidecar.py` shows the verification function
- `uv run --python 3.13 pytest tests/test_midnight_preflight.py -v` — all 6 tests pass
</verification>

<success_criteria>

- ClickHouse query for yesterday's orphan bar trade ID exists and handles errors gracefully
- Binance REST query for midnight crossover trade ID exists and handles errors gracefully
- Verification function logs INFO on match, WARNING on mismatch/missing, never crashes
- All 6 unit tests pass
  </success_criteria>

<output>
After completion, create `.planning/phases/02-midnight-preflight-verification/02-01-SUMMARY.md`
</output>
