# REST→WS Junction Fix + Full-Symbol Streaming

## What This Is

A comprehensive fix for the forming-bar abandonment bug at the REST→WS junction in the streaming sidecar, combined with expanding streaming to all 18 symbols via combined WebSocket streams. When the sidecar starts, `fill_from_rest()` processes symbols sequentially (~5 min). During this time, WS buffers trades. At the junction, Puck's gap detector fires on the expected gap, disrupting the forming bar and creating multi-hour data gaps (up to 12h, 100K trades). This project fixes the junction, adds telemetry, parallelizes REST fill, implements combined WS streams, and deploys with zero-tolerance Stathera verification.

## Core Value

Zero intra-day Stathera gaps and zero overlaps across all 18 symbols after sidecar restart — the streaming pipeline must produce bit-exact continuous trade-ID coverage without manual intervention.

## Requirements

### Validated

- ✓ Sidecar streams symbols via per-symbol WS connections — existing
- ✓ fill_from_rest catches up from checkpoints via sequential REST fetch — existing
- ✓ Puck inline gap detection fills small gaps in real-time — existing
- ✓ Day-mode invariant enforced (L1-L3 cross-midnight prevention) — v13.10.0
- ✓ Writer ownership model (sidecar=today, kintsugi=yesterday+older) — v13.5.2+
- ✓ Clean-slate deploy playbook documented — existing skill

### Active

- [ ] Junction fix: suppress Puck and preserve forming bar at REST→WS transition
- [ ] Junction telemetry: forming bar trade count, WS buffer depth, Puck suppression events
- [ ] Parallel fill_from_rest with partitioned rate limiter (6000 weight/min ÷ N)
- [ ] Combined WS streams: spike-test 1×18, 2×9, and tiered (2+16) configurations
- [ ] Combined WS streams: implement winning configuration in Rust
- [ ] All 5 audit items from root cause analysis (ring buffer, per-symbol timing, buffer depth, Puck junction logging, forming bar preservation metric)
- [ ] Linux wheel build (Zig cross-compile)
- [ ] Clean-slate deploy to bigblack (Stop → Delete today → Clear checkpoints → Start)
- [ ] Zero-tolerance Stathera verification: 0 intra-day gaps, 0 overlaps, all 18 symbols, 24h+

### Out of Scope

- Kintsugi changes — writer ownership model is stable, no changes needed
- Asclepius reactivation — permanently disabled (#284), out of scope
- Heartbeat/monitoring changes — existing telemetry is sufficient
- New symbol onboarding — current 18 symbols are the target
- Unified engine (Issue #178) — separate initiative

## Context

**Root cause proven 2026-03-19** with full Rust tracing on SOLUSDT@500: fill_from_rest fetched 149,405 trades, processor produced 9 bars + forming bar with ~100K accumulated trades. At junction, Puck fired, disrupting the forming bar. Result: 99,882-trade gap between bar #9 (07:13) and bar #10 (19:05).

**Secondary problem**: Only BTCUSDT/ETHUSDT are currently streamed. Other 16 symbols rely on kintsugi/backfill joins at day boundaries, creating persistent midnight gaps proportional to threshold. Expanding to all 18 symbols via combined streams eliminates this entire class of gaps.

**Combined WS empirical data**: 2×9 hybrid previously tested at 138.7 trades/s with zero recv timeouts. Binance allows max 1024 streams/connection, 5 connections/IP. All 18 symbols need only 18 streams — well within limits.

**Fix location**: `crates/opendeviationbar-streaming/src/live_engine.rs` `symbol_task` function, between processor reuse and main `loop { select! { ... } }`.

## Constraints

- **Rate limits**: Binance REST 6000 weight/min — parallel fill must partition this budget
- **Python 3.13 only**: No other Python versions
- **Spot only**: All data from Binance spot market
- **bigblack deploy**: Production is a single GPU host, systemd services
- **Zero tolerance**: 0 gaps, 0 overlaps across all 18 symbols for 24h+ post-deploy

## Key Decisions

| Decision                                         | Rationale                                                                           | Outcome   |
| ------------------------------------------------ | ----------------------------------------------------------------------------------- | --------- |
| Suppress Puck at junction (not disable globally) | Puck is valuable during normal streaming; only the initial junction gap is expected | — Pending |
| Partitioned rate limiter for parallel fill       | Divide 6000 weight/min across concurrent fills — predictable per-symbol budget      | — Pending |
| Spike-test 3 WS configs (1×18, 2×9, tiered)      | 2×9 was empirically best but other configs untested. Data-driven selection.         | — Pending |
| Clean-slate deploy (not rolling)                 | Junction fix changes startup behavior; clean state needed for valid verification    | — Pending |

---

_Last updated: 2026-03-19 after initialization_
