# Phase 2: Sidecar Fill Pipeline - Context

**Gathered:** 2026-03-24
**Status:** Ready for planning
**Mode:** Auto-generated (infrastructure phase — smart discuss skipped)

<domain>
## Phase Boundary

Sidecar startup produces zero-gap bar coverage from midnight to WS junction without intermediate trade hand-off. `fill_from_rest` returns computed bars directly for immediate `sync_flush` to ClickHouse. Stathera continuity guaranteed at the REST→WS boundary.

</domain>

<decisions>
## Implementation Decisions

### Claude's Discretion

All implementation choices are at Claude's discretion — pure infrastructure phase. Use ROADMAP phase goal, success criteria, and codebase conventions to guide decisions.

Key technical constraints from CLAUDE.md:

- `fill_from_rest` currently lives in `sidecar/midnight.py` and is called from `sidecar/__init__.py:365`
- `_sync_flush_rest_bars` in `midnight.py:641` handles the CH write path
- Trades must stay in Rust (`OpenDeviationBarProcessor.process_trades()`) — no Python DataFrame intermediary
- Stathera continuity: `bar[last_rest].last_agg_trade_id + 1 == bar[first_ws].first_agg_trade_id`
- Writer ownership: sidecar owns today, kintsugi owns yesterday+older
- Clean-slate DELETE today runs BEFORE fill_from_rest (Step 3 of startup sequence)

### Carrying forward from Phase 1

- Per-partition locks now available in `maintenance.py` — any writes during fill_from_rest benefit from concurrent partition access
- All DELETEs use `DELETE IN PARTITION` — no mutation timeout risk during startup

</decisions>

<canonical_refs>

## Canonical References

**Downstream agents MUST read these before planning or implementing.**

### Sidecar architecture

- `python/opendeviationbar/sidecar/CLAUDE.md` — sidecar module map, startup sequence, midnight preflight
- `python/opendeviationbar/sidecar/__init__.py` — `_create_engine()` startup sequence (Steps 1-6 in CLAUDE.md)
- `python/opendeviationbar/sidecar/midnight.py` — `fill_from_rest()`, `_sync_flush_rest_bars()`, `_run_midnight_preflight()`

### Core algorithm

- `CLAUDE.md` §6 "Sidecar Startup Sequence Invariant" — exact ordering of startup steps
- `crates/CLAUDE.md` — Rust processor API, `process_trades()`, checkpoint API
- `python/opendeviationbar/sidecar/clickhouse_write.py` — CH write path from sidecar

### Data integrity

- `CLAUDE.md` §"Stathera" terminology — trade-ID alignment invariant
- `python/opendeviationbar/validation/day_mode/CLAUDE.md` — day-mode invariant enforcement

</canonical_refs>

<code_context>

## Existing Code Insights

### Reusable Assets

- `_sync_flush_rest_bars()` in `midnight.py:641` — already handles batch write of bars to CH
- `OpenDeviationBarProcessor.process_trades()` — Rust-side bar computation, maintains state
- `store_bars_batch(overlap_strategy="replace")` with per-partition locks (Phase 1)

### Established Patterns

- Sidecar startup sequence: WS start → clean-slate DELETE → midnight preflight → fill_from_rest → sync_flush
- `fill_from_rest` currently returns bars via `engine.fill_from_rest()` at `__init__.py:365`
- `committed_floors` initialized from `processor.last_completed_bar_tid()` for dedup

### Integration Points

- `__init__.py:365` — call site for `fill_from_rest()`
- `midnight.py` — implementation of fill_from_rest and sync_flush
- REST→WS junction: `fill_from_rest` bars hand off to WS streaming at the trade-ID boundary

</code_context>

<specifics>
## Specific Ideas

No specific requirements — infrastructure phase. Refer to ROADMAP phase description and success criteria.

</specifics>

<deferred>
## Deferred Ideas

None — infrastructure phase.

</deferred>

---

_Phase: 02-sidecar-fill-pipeline_
_Context gathered: 2026-03-24_
