# Phase 30: Parquet-First Gap Fill + Junction Telemetry - Context

**Gathered:** 2026-03-28
**Status:** Ready for planning
**Mode:** Auto-generated (infrastructure phase — discuss skipped)

<domain>
## Phase Boundary

Sidecar fills restart gaps from local Parquet tick cache before falling back to REST API, and logs junction quality metrics at the REST-to-WS handoff point. Currently `fill_from_rest` goes straight to Binance REST API for all gap-fill trades. The seeder maintains `~/.cache/opendeviationbar/ticks/BINANCE_SPOT_{SYMBOL}/YYYY-MM-DD.parquet` for all symbols — these files should be the first source (3ms/file vs REST round-trips).

</domain>

<decisions>
## Implementation Decisions

### Claude's Discretion

All implementation choices are at Claude's discretion — pure infrastructure phase. Use ROADMAP phase goal, success criteria, and codebase conventions to guide decisions.

Key context:

- `fill_from_rest` is a Rust function on the StreamManager — it fetches trades via REST API
- The Parquet-first layer must be PYTHON-side: read Parquet files, feed trades to the engine BEFORE calling fill_from_rest
- Use `TickStorage` from `opendeviationbar.storage.parquet` — `has_ticks()` and `read_ticks()` methods
- Parquet files are daily: `YYYY-MM-DD.parquet` with columns including `agg_trade_id`
- The gap range is from `resume_tid + 1` to current time — may span multiple days
- Junction telemetry is already partially logged by Rust (`junction.telemetry` log line) — Python should log additional metrics
- RESUME-02: Parquet-first fill. RESUME-03: Junction telemetry.

</decisions>

<code_context>

## Existing Code Insights

### Key Files

- `python/opendeviationbar/sidecar/__init__.py` — `_create_engine()` Aion startup with fill_from_rest call
- `python/opendeviationbar/sidecar/midnight.py` — `fill_from_rest()` dispatch, `_sync_flush_rest_bars()`
- `python/opendeviationbar/storage/parquet.py` — `TickStorage` class with `has_ticks()`, `read_ticks()`
- `python/opendeviationbar/sidecar/delete_strategy.py` — Phase 29 output
- `python/opendeviationbar/sidecar/exchange_gap_floor.py` — Phase 29 output

### Parquet-First Pattern (from kintsugi)

Kintsugi already does Parquet-first in `_repair_aion()`:

```python
storage = TickStorage()
if storage.has_ticks(symbol, start_ms, end_ms):
    trades = storage.read_ticks(symbol, start_ms, end_ms)
    # Process trades through Rust processor
```

### fill_from_rest Architecture

`fill_from_rest()` is Rust-native — it fetches REST trades internally and feeds them to processors. To add a Parquet-first layer, we need a PYTHON pre-fill step that:

1. Reads Parquet trades for the gap range
2. Feeds them to the engine via `engine.process_trades()` or similar
3. Only calls `fill_from_rest()` for the remaining uncovered range (if any)

</code_context>

<specifics>
## Specific Ideas

No specific requirements — infrastructure phase.

</specifics>

<deferred>
## Deferred Ideas

None — infrastructure phase.

</deferred>
