# Requirements: REST→WS Junction Fix + Full-Symbol Streaming

**Defined:** 2026-03-19
**Core Value:** Zero intra-day Stathera gaps and zero overlaps across all 18 symbols after sidecar restart

## v1 Requirements

### Junction Fix

- [x] **JUNC-01**: Sidecar preserves forming bar across REST→WS transition — no trade-ID gaps from Puck misfiring at the expected junction gap
- [x] **JUNC-02**: Junction fill uses the SAME processor instance (single-processor continuity invariant)
- [x] **JUNC-03**: Gap detector re-seeded past junction before entering main WS loop — Puck suppressed for the initial expected gap only
- [x] **JUNC-04**: Junction fill handles edge case where WS buffer has trades overlapping with REST's last trade (monotonicity guard)

### Telemetry

- [ ] **TELE-01**: Log junction gap size (trade-ID delta between REST last and WS first) per symbol at startup
- [ ] **TELE-02**: Log forming bar trade count at junction transition per symbol
- [ ] **TELE-03**: Log WS buffer depth (pending trades) per symbol when engine starts
- [ ] **TELE-04**: Log whether Puck was suppressed at junction vs fired during normal streaming (distinct event types)
- [ ] **TELE-05**: Log per-symbol fill_from_rest duration, trade count, and bars produced

### Parallel Backfill

- [ ] **PARA-01**: fill_from_rest processes all 18 symbols concurrently (bounded by Semaphore) instead of sequentially
- [ ] **PARA-02**: Total fill time reduced from ~5 min to <90s (4 concurrent workers)
- [ ] **PARA-03**: Rate limiter dynamic ceiling: 4800 weight/min during fill, 3000 after start (reserves headroom for Puck + kintsugi)
- [ ] **PARA-04**: WS channel capacity increased from 1000 to 50,000 to prevent buffer overflow during parallel fill

### Combined WS Streams

- [ ] **COMB-01**: Spike-test 3 configurations (1×18, 2×9 even, 2×9 volume-tiered) — measure throughput, reconnect latency, gap count per config
- [ ] **COMB-02**: Implement winning configuration as new `CombinedWebSocketStream` struct with per-symbol demux
- [ ] **COMB-03**: Combined stream message envelope parsing (`{"stream":"...","data":{...}}` format)
- [ ] **COMB-04**: All 18 symbols streamed via combined connections (eliminate per-symbol WS connections)
- [ ] **COMB-05**: Overlapping reconnection at 23h — open new connection, stabilize, close old. Stagger 2 connections by 30 min.
- [ ] **COMB-06**: `symbol_task` interface unchanged — combined streams output to same per-symbol channels

### Audit Items

- [x] **AUDT-01**: Verify ring buffer capacity (10K) sufficient for 18 symbols × all thresholds during full-day fill
- [ ] **AUDT-02**: Per-symbol REST fill timing with per-bar-per-threshold breakdown in Rust tracing
- [ ] **AUDT-03**: WS buffer depth logging when engine starts (per-symbol channel pending count)
- [ ] **AUDT-04**: Puck junction vs normal-streaming event differentiation in logs
- [ ] **AUDT-05**: Forming bar preservation metric — forming bar trade count logged if >0 at junction

### Deployment

- [ ] **DEPL-01**: Linux x86_64 wheel built via Zig cross-compile
- [ ] **DEPL-02**: Backup today's bars to `_backup` table before clean-slate delete
- [ ] **DEPL-03**: Clean-slate deploy: Stop → Delete today → Clear checkpoints → Start
- [ ] **DEPL-04**: Deploy during UTC 00:00-02:00 low-volume window
- [ ] **DEPL-05**: Rollback SQL script prepared and tested before deploy

### Verification

- [ ] **VERF-01**: 0 intra-day Stathera gaps across all 18 symbols for 24h+ post-deploy
- [ ] **VERF-02**: 0 Stathera overlaps across all 18 symbols for 24h+ post-deploy
- [ ] **VERF-03**: All 18 symbols producing bars via combined WS streams (not per-symbol connections)
- [ ] **VERF-04**: Junction telemetry confirms forming bar preservation at every sidecar restart
- [ ] **VERF-05**: Rate limiter headroom maintained (never exceeds 5000 weight/min during normal operation)

## v2 Requirements

### Performance Optimization

- **PERF-01**: Upgrade tokio-tungstenite to 0.28 after combined streams are stable (2+ weeks)
- **PERF-02**: Prometheus metrics endpoint for real-time dashboard
- **PERF-03**: Adaptive parallelism for fill_from_rest (scale workers based on rate limiter headroom)

### Resilience

- **RESL-01**: Kintsugi rate limiter coordination (Issue #291) — shared awareness of IP-level REST budget
- **RESL-02**: Dynamic combined stream group rebalancing based on volume changes
- **RESL-03**: Hot checkpoint persistence — checkpoints saved every N bars, not just at shutdown

## Out of Scope

| Feature                      | Reason                                                           |
| ---------------------------- | ---------------------------------------------------------------- |
| Kintsugi changes             | Writer ownership model is stable; no changes needed for this fix |
| Asclepius reactivation       | Permanently disabled (#284); unsafe for old gaps on live engine  |
| Heartbeat/monitoring changes | Existing telemetry is sufficient; new metrics go to journal      |
| New symbol onboarding        | Current 18 symbols are the target set                            |
| Unified engine (Issue #178)  | Separate initiative, phases 4-7 pending                          |
| Futures market data          | Spot-only policy is firm                                         |
| Per-trade latency tracking   | Noise at this scale, anti-feature per research                   |
| Full trade replay logging    | Storage cost prohibitive, anti-feature per research              |

## Traceability

| Requirement | Phase                              | Status  |
| ----------- | ---------------------------------- | ------- |
| JUNC-01     | Phase 1 — Junction Fix             | Complete |
| JUNC-02     | Phase 1 — Junction Fix             | Complete |
| JUNC-03     | Phase 1 — Junction Fix             | Complete |
| JUNC-04     | Phase 1 — Junction Fix             | Complete |
| TELE-01     | Phase 2 — Telemetry + Audit        | Pending |
| TELE-02     | Phase 2 — Telemetry + Audit        | Pending |
| TELE-03     | Phase 2 — Telemetry + Audit        | Pending |
| TELE-04     | Phase 2 — Telemetry + Audit        | Pending |
| TELE-05     | Phase 2 — Telemetry + Audit        | Pending |
| PARA-03     | Phase 3 — Rate Limiter Ceiling     | Pending |
| PARA-01     | Phase 4 — Parallel fill_from_rest  | Pending |
| PARA-02     | Phase 4 — Parallel fill_from_rest  | Pending |
| PARA-04     | Phase 4 — Parallel fill_from_rest  | Pending |
| COMB-01     | Phase 5 — Combined WS Spike Test   | Pending |
| COMB-02     | Phase 6 — Combined WS Impl         | Pending |
| COMB-03     | Phase 6 — Combined WS Impl         | Pending |
| COMB-04     | Phase 6 — Combined WS Impl         | Pending |
| COMB-06     | Phase 6 — Combined WS Impl         | Pending |
| COMB-05     | Phase 7 — Overlapping Reconnection | Pending |
| AUDT-01     | Phase 2 — Telemetry + Audit        | Complete |
| AUDT-02     | Phase 2 — Telemetry + Audit        | Pending |
| AUDT-03     | Phase 2 — Telemetry + Audit        | Pending |
| AUDT-04     | Phase 2 — Telemetry + Audit        | Pending |
| AUDT-05     | Phase 2 — Telemetry + Audit        | Pending |
| DEPL-01     | Phase 9 — Build + Deploy Prep      | Pending |
| DEPL-02     | Phase 9 — Build + Deploy Prep      | Pending |
| DEPL-03     | Phase 9 — Build + Deploy Prep      | Pending |
| DEPL-04     | Phase 9 — Build + Deploy Prep      | Pending |
| DEPL-05     | Phase 9 — Build + Deploy Prep      | Pending |
| VERF-01     | Phase 10 — Deploy + Verification   | Pending |
| VERF-02     | Phase 10 — Deploy + Verification   | Pending |
| VERF-03     | Phase 10 — Deploy + Verification   | Pending |
| VERF-04     | Phase 10 — Deploy + Verification   | Pending |
| VERF-05     | Phase 10 — Deploy + Verification   | Pending |

**Coverage:**

- v1 requirements: 30 total
- Mapped to phases: 30/30
- Unmapped: 0
- Phase 8 (Integration Testing): no new requirements — validates Phases 1-7 together

---

_Requirements defined: 2026-03-19_
_Last updated: 2026-03-19 — roadmap phase assignments finalized_
