# Requirements Archive: v1.4 Zero-Gap Zero-Overlap Guarantee

**Archived:** 2026-03-23
**Status:** SHIPPED

For current requirements, see `.planning/REQUIREMENTS.md`.

---

# Requirements: v1.4 -- Zero-Gap Zero-Overlap Guarantee

**Defined:** 2026-03-21
**Core Value:** Zero Stathera gaps and overlaps across every restart, every midnight boundary, every deploy -- verified by continuous telemetry.

## v1 Requirements

### Rust Core (committed_floors fix)

- [x] **RUST-01**: `committed_floors` in `symbol_task()` initialized from last COMPLETED bar's `last_agg_trade_id`, not from `processor.last_agg_trade_id()` (which includes forming bar tail)
- [x] **RUST-02**: After fill_from_rest -> WS junction, zero bars are suppressed by the dedup floor -- verified by Stathera audit showing zero intra-day gaps

### Midnight Preflight Verification

- [x] **PREFLIGHT-01**: On checkpoint-cleared restart, sidecar queries Binance REST API for the exact midnight crossover trade ID (`startTime=midnight_ms&limit=1`) for each streaming symbol
- [x] **PREFLIGHT-02**: Sidecar verifies yesterday's orphan bar in ClickHouse closes at `crossover_trade_id - 1`. If not, logs WARNING and defers to kintsugi (no data corruption)
- [x] **PREFLIGHT-03**: `fill_from_rest` anchors from the verified crossover trade ID (not CH seed), guaranteeing `yesterday_orphan.last_agg_trade_id + 1 == today_first_bar.first_agg_trade_id`

### Heartbeat Telemetry

- [x] **TELEM-01**: Heartbeat reports per-symbol Stathera continuity as a single unified check -- no separate "intra-day" vs "midnight-boundary" classification. Gap is a gap.
- [x] **TELEM-02**: Heartbeat reports yesterday completion status per symbol -- shows whether yesterday's orphan bar matches the midnight crossover trade ID
- [x] **TELEM-03**: After sidecar restart, heartbeat reports junction telemetry: REST last trade ID, WS first trade ID, forming bar state, committed_floors value
- [x] **TELEM-04**: If any new gap appears in today's bars, heartbeat sends LOUD Telegram alert with exact (symbol, threshold, tid_delta, trade ID range)

### Seeder REST Integration & Rate Limit Coordination

- [x] **SEEDER-01**: Seeder attempts REST API fallback for days where Vision CSV is not yet available (typically yesterday), using the same `aggTrades` endpoint as kintsugi
- [x] **SEEDER-02**: All three REST consumers (sidecar Puck/fill_from_rest, kintsugi, seeder) share rate limit state via a common mechanism (file-based), preventing any single consumer from starving others
- [x] **SEEDER-03**: Total REST API weight consumption across all processes stays below 80% of Binance's 6000 weight/min IP-wide budget under normal operation
- [x] **SEEDER-04**: Seeder REST path respects writer ownership model: only seeds Parquet ticks (no ClickHouse writes), only for days kintsugi hasn't yet processed

### Memory Efficiency Refactoring (P0 Findings from Issue #300)

- [x] **MEM-01**: `AggTrade` struct derives `Copy` — eliminates clone trait dispatch on hot path where every trade is cloned for intra-bar feature accumulation
- [x] **MEM-02**: Arrow iterator adapter replaces `Vec<AggTrade>` materialization in `record_batch_to_aggtrades()` — stack-local AggTrade per iteration, no heap Vec
- [x] **MEM-03**: GIL released during pure Rust computation in batch `process_trades_arrow()` and `process_trades_arrow_native()` via `py.allow_threads()`
- [x] **MEM-04**: Plugin enrichment skips Polars-to-Pandas roundtrip when no plugins installed (the common case), eliminating 2x flush peak memory
- [x] **MEM-05**: `get_stats()` / `get_cache_stats()` uses Parquet footer metadata for row counting instead of materializing 12-42 GB of data
- [x] **MEM-06**: REST fallback path in `backfill.py` uses Arrow streaming instead of `list[dict]` materialization, saving 400-600 MB/day
- [x] **MEM-07**: `store_bars_batch()` merges all `with_columns()` calls into a single call, eliminating 5+ intermediate DataFrame allocations per write
- [x] **MEM-08**: FormingBar watch channel updates throttled to 1 Hz, eliminating ~99.3% of unnecessary OpenDeviationBar clones (~520B each at 560/sec)

## v2 Requirements

- **STREAM-01**: Expand sidecar streaming to all 17 symbols (eliminates multi-processor seam entirely)
- **KINTSUGI-01**: Two-day processor spans in kintsugi (alternative to preflight for non-streamed symbols)

## Out of Scope

| Feature                       | Reason                                           |
| ----------------------------- | ------------------------------------------------ |
| Full 18-symbol streaming      | Planned for future milestone (GSD Phases 6-10)   |
| Kintsugi two-day spans        | Preflight verification is simpler and sufficient |
| OpenDeviationBar struct split | Major refactor, separate milestone               |
| Plugin Polars->Pandas         | No plugins in production                         |

## Traceability

| Requirement  | Phase   | Status      |
| ------------ | ------- | ----------- |
| RUST-01      | Phase 1 | Complete    |
| RUST-02      | Phase 1 | Complete    |
| PREFLIGHT-01 | Phase 2 | Complete    |
| PREFLIGHT-02 | Phase 2 | Complete    |
| PREFLIGHT-03 | Phase 2 | Complete    |
| TELEM-01     | Phase 3 | Complete    |
| TELEM-02     | Phase 3 | Complete    |
| TELEM-03     | Phase 3 | Complete    |
| TELEM-04     | Phase 3 | Complete    |
| SEEDER-01    | Phase 5 | Complete    |
| SEEDER-02    | Phase 5 | Complete    |
| SEEDER-03    | Phase 5 | Complete    |
| SEEDER-04    | Phase 5 | Complete    |
| MEM-01       | Phase 6 | Not started |
| MEM-02       | Phase 6 | Not started |
| MEM-03       | Phase 6 | Not started |
| MEM-04       | Phase 6 | Not started |
| MEM-05       | Phase 6 | Not started |
| MEM-06       | Phase 6 | Not started |
| MEM-07       | Phase 6 | Not started |
| MEM-08       | Phase 6 | Not started |

**Coverage:**

- v1 requirements: 21 total
- Mapped to phases: 21
- Unmapped: 0

---

_Requirements defined: 2026-03-21_
_Updated: 2026-03-22 (added SEEDER-01 through SEEDER-04 for Phase 5)_
_Updated: 2026-03-23 (added MEM-01 through MEM-08 for Phase 6)_
_Derived from: 6-agent spike (2026-03-21) + user architecture decisions + Issue #300 9-agent memory audit_
