---
phase: 02-midnight-preflight-verification
plan: 01
subsystem: streaming
tags: [sidecar, clickhouse, binance-rest, stathera, midnight, preflight]

# Dependency graph
requires:
  - phase: 01-dedup-floor-fix
    provides: last_completed_bar_tid tracking for dedup floor continuity
provides:
  - _get_midnight_crossover_tid() function in sidecar.py
  - _verify_yesterday_orphan() function in sidecar.py
  - get_yesterday_orphan_tid() ClickHouse query method
  - Unit tests for all preflight functions
affects: [02-02-PLAN, heartbeat-telemetry, sidecar-startup]

# Tech tracking
tech-stack:
  added: []
  patterns: [stamina-retry-inside-function, never-raise-preflight-pattern]

key-files:
  created:
    - tests/test_midnight_preflight.py
  modified:
    - python/opendeviationbar/clickhouse/query_operations.py
    - python/opendeviationbar/sidecar.py

key-decisions:
  - "Preflight functions never raise -- return None/False on any error, deferring to kintsugi"
  - "1-second REST window (midnight_ms + 1000) sufficient to capture first trade after midnight"
  - "stamina retry (3 attempts, 30s timeout) for REST resilience inside _get_midnight_crossover_tid"

patterns-established:
  - "Never-raise preflight: functions return safe defaults and log warnings, never crash sidecar startup"

requirements-completed: [PREFLIGHT-01, PREFLIGHT-02]

# Metrics
duration: 2min
completed: 2026-03-21
---

# Phase 02 Plan 01: Midnight Preflight Query Functions Summary

**Binance REST midnight crossover query and ClickHouse orphan verification for Stathera continuity at day boundaries**

## Performance

- **Duration:** 2 min
- **Started:** 2026-03-21T14:35:36Z
- **Completed:** 2026-03-21T14:37:43Z
- **Tasks:** 2
- **Files modified:** 3

## Accomplishments

- ClickHouse query method `get_yesterday_orphan_tid()` to find yesterday's orphan bar trade ID
- `_get_midnight_crossover_tid()` queries Binance REST for exact first trade after UTC midnight
- `_verify_yesterday_orphan()` checks orphan bar closes at crossover_tid - 1 (Stathera continuity)
- 6 unit tests covering success, mismatch, missing orphan, and network error paths

## Task Commits

Each task was committed atomically:

1. **Task 1: Add ClickHouse query for yesterday's orphan bar trade ID** - `17b5a55` (feat)
2. **Task 2: Add \_get_midnight_crossover_tid and \_verify_yesterday_orphan to sidecar.py** - `bfe6f1f` (feat)

## Files Created/Modified

- `python/opendeviationbar/clickhouse/query_operations.py` - Added get_yesterday_orphan_tid() method to QueryOperationsMixin
- `python/opendeviationbar/sidecar.py` - Added \_get_midnight_crossover_tid() and \_verify_yesterday_orphan() functions
- `tests/test_midnight_preflight.py` - 6 unit tests for midnight preflight functions

## Decisions Made

- Preflight functions never raise -- they return None/False on any error, logging warnings and deferring to kintsugi for gap repair
- Used 1-second REST window (midnight_ms to midnight_ms + 1000) to capture first trade after midnight with minimal data transfer
- stamina retry decorator (3 attempts, 30s timeout) applied inside function scope, following existing sidecar.py patterns

## Deviations from Plan

### Auto-fixed Issues

**1. [Rule 1 - Bug] Fixed mock path in test_get_midnight_crossover_tid_success**

- **Found during:** Task 2 (test execution)
- **Issue:** Test patched `opendeviationbar.sidecar.fetch_aggtrades_rest` but the import is inside the function from `opendeviationbar._core`
- **Fix:** Changed mock target to `opendeviationbar._core.fetch_aggtrades_rest`
- **Files modified:** tests/test_midnight_preflight.py
- **Verification:** All 6 tests pass
- **Committed in:** bfe6f1f (Task 2 commit)

---

**Total deviations:** 1 auto-fixed (1 bug)
**Impact on plan:** Minor test mock path fix. No scope creep.

## Issues Encountered

None

## User Setup Required

None - no external service configuration required.

## Next Phase Readiness

- Both preflight query functions are ready for Plan 02 to wire into \_create_engine() startup path
- Functions are self-contained and tested, Plan 02 will orchestrate calling them during sidecar startup

---

_Phase: 02-midnight-preflight-verification_
_Completed: 2026-03-21_
