---
phase: 04-combined-websocket-streams
verified: 2026-03-24T21:02:22Z
status: human_needed
score: 3/4 success criteria verified automatically
human_verification:
  - test: "Deploy sidecar to bigblack with OPENDEVIATIONBAR_STREAMING_WS_MODE=combined and run for 24+ hours"
    expected: "Zero Stathera gaps across all 26 symbols after migration to combined streams; /health endpoint shows reconnections=0 or only expected proactive 23h reconnections"
    why_human: "WS-03 Stathera continuity requires a live production run; cannot be verified statically. Success criterion 4 (WS connection count observable in /health) also needs runtime observation."
  - test: "Check /health endpoint after sidecar starts in combined mode"
    expected: "Health endpoint response shows ws_mode or otherwise confirms 2 WS connections active instead of 18+"
    why_human: "The /health endpoint does not currently expose ws_mode or active WS connection count. Observability of success criterion 4 cannot be confirmed without a live run."
---

# Phase 4: Combined WebSocket Streams Verification Report

**Phase Goal:** All symbols stream via Binance combined streams with resilient reconnection maintaining Stathera continuity
**Verified:** 2026-03-24T21:02:22Z
**Status:** human_needed
**Re-verification:** No — initial verification

## Goal Achievement

### Observable Truths (from PLAN 04-01 must_haves)

| #   | Truth                                                                                                                                              | Status     | Evidence                                                                                                                                                                                                                                              |
| --- | -------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 1   | Setting `OPENDEVIATIONBAR_WS_MODE=per_symbol` causes sidecar to spawn N individual `BinanceWebSocketStream` connections                            | ✓ VERIFIED | `start_ws()` WsMode::PerSymbol branch spawns `BinanceWebSocketStream::run_with_reconnect` per symbol (mod.rs line 254-280). Smoke test: env var plumbed through SidecarConfig                                                                         |
| 2   | Setting `OPENDEVIATIONBAR_WS_MODE=combined` (default) causes sidecar to spawn 2 `CombinedWebSocketStream` connections with symbols split by volume | ✓ VERIFIED | `start_ws()` WsMode::Combined branch calls `split_symbols_by_volume()` and spawns two `CombinedWebSocketStream::run_with_reconnect` tasks (mod.rs lines 194-252). Log confirms "started 2 combined WS connections"                                    |
| 3   | Both WS modes produce identical per-symbol `mpsc::Receiver<AggTrade>` channels consumed by symbol_task unchanged                                   | ✓ VERIFIED | Both branches populate the same `receivers` HashMap. `ws_trade_receivers` set identically at end of both branches. symbol_task code unchanged (pre_buffered_rx consumed identically)                                                                  |
| 4   | Antaeus exponential backoff with jitter applies to both combined connections independently                                                         | ✓ VERIFIED | Each `CombinedWebSocketStream::run_with_reconnect` call receives its own `child_token()` and cloned `policy`. `combined_ws.rs:build_backoff()` uses `ExponentialBuilder::with_jitter()`. Independent state per connection confirmed by code isolation |

**Score:** 4/4 must_have truths verified

### Success Criteria (from ROADMAP.md)

| #    | Success Criterion                                                                                          | Status         | Evidence                                                                                                                                                                                                                                                               |
| ---- | ---------------------------------------------------------------------------------------------------------- | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| SC-1 | Sidecar connects via `/stream?streams=...` using 2x9 hybrid architecture instead of per-symbol connections | ✓ VERIFIED     | Combined WS uses Binance `/stream?streams=` URL (combined_ws.rs). Split is 2x13 (not 2x9) — research doc explicitly updated scope: "2x13 given 26 symbols, not 2x9 as originally scoped for 18 symbols". 26 symbols are enabled and split 13+13 by volume ranking      |
| SC-2 | Combined stream reconnection uses Antaeus exponential backoff with jitter                                  | ✓ VERIFIED     | `combined_ws.rs:build_backoff()` uses `backon::ExponentialBuilder::with_jitter()`. Each connection has independent policy clone and child CancellationToken                                                                                                            |
| SC-3 | All 26 symbols maintain Stathera continuity (zero gaps, zero overlaps) for 24+ hours after migration       | ? HUMAN NEEDED | Code-level correctness verified (mode-agnostic mpsc channels, independent connections). Production 24h+ run required to confirm zero Stathera gaps                                                                                                                     |
| SC-4 | Total WS connection count drops from 18+ to 2 (observable in sidecar /health endpoint)                     | ? HUMAN NEEDED | Code spawns exactly 2 tasks in Combined mode. However `/health` endpoint does not expose `ws_mode` or an active WS connection count field — only `connection_count` (cumulative CONNECT signal counter). Observability of "2 connections" via `/health` is not present |

**Score:** 2/4 success criteria need human verification (SC-3: live production run; SC-4: /health observability gap)

### Required Artifacts

| Artifact                                                     | Expected                                               | Status     | Details                                                                                                                                                                                                                   |
| ------------------------------------------------------------ | ------------------------------------------------------ | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `crates/opendeviationbar-streaming/src/live_engine/types.rs` | WsMode enum and ws_mode field on LiveEngineConfig      | ✓ VERIFIED | WsMode enum (lines 87-92), ws_mode field on LiveEngineConfig (line 173), split_symbols_by_volume (lines 102-141), 5 unit tests (lines 276-357)                                                                            |
| `crates/opendeviationbar-streaming/src/live_engine/mod.rs`   | start_ws() branching on WsMode with 2-connection split | ✓ VERIFIED | WsMode::Combined branch (lines 194-253), WsMode::PerSymbol branch (lines 254-280), BinanceWebSocketStream import (line 34), split_symbols_by_volume called (line 203)                                                     |
| `python/opendeviationbar/config/sidecar.py`                  | ws_mode config field with env var                      | ✓ VERIFIED | `ws_mode: Literal["combined", "per_symbol"] = "combined"` at line 201. Env var: OPENDEVIATIONBAR*STREAMING_WS_MODE (via env_prefix="OPENDEVIATIONBAR_STREAMING*")                                                         |
| `src/stream_manager_bindings.rs`                             | ws_mode parameter threaded from Python to Rust         | ✓ VERIFIED | PyO3 signature includes `ws_mode="combined"` default (line 96), match arm maps string to WsMode enum (lines 125-128). Runtime verified: `StreamManager(['BTCUSDT'], [250], ws_mode='combined')` instantiates successfully |

### Key Link Verification

| From                                                         | To                                                           | Via                                         | Status  | Details                                                                                              |
| ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------- | ------- | ---------------------------------------------------------------------------------------------------- |
| `python/opendeviationbar/config/sidecar.py`                  | `python/opendeviationbar/sidecar/__init__.py`                | SidecarConfig.ws_mode read in_create_engine | ✓ WIRED | `ws_mode=config.ws_mode` passed to StreamManager at **init**.py line 219                             |
| `python/opendeviationbar/sidecar/__init__.py`                | `src/stream_manager_bindings.rs`                             | StreamManager constructor ws_mode parameter | ✓ WIRED | PyO3 accepts `ws_mode` kwarg; runtime smoke test confirmed                                           |
| `src/stream_manager_bindings.rs`                             | `crates/opendeviationbar-streaming/src/live_engine/types.rs` | LiveEngineConfig.ws_mode field              | ✓ WIRED | `engine_config.ws_mode = match ws_mode { ... }` at bindings.rs line 125                              |
| `crates/opendeviationbar-streaming/src/live_engine/types.rs` | `crates/opendeviationbar-streaming/src/live_engine/mod.rs`   | start_ws() reads self.config.ws_mode        | ✓ WIRED | `match self.config.ws_mode { WsMode::Combined => ..., WsMode::PerSymbol => ... }` at mod.rs line 193 |

### Data-Flow Trace (Level 4)

Not applicable — this phase adds control-flow branching, not data rendering. The WS connections are infrastructure (no data rendered to a UI). The mpsc channel wiring is verified at Level 3.

### Behavioral Spot-Checks

| Behavior                                                      | Command                                                                                                                                                            | Result                                    | Status             |
| ------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------- | ------------------ | ------ |
| ws_mode defaults to "combined"                                | `python3 -c "from opendeviationbar.config.sidecar import SidecarConfig; c = SidecarConfig(symbols=['BTCUSDT'], thresholds=[250]); assert c.ws_mode == 'combined'"` | OK: ws_mode defaults to combined          | ✓ PASS             |
| Env var OPENDEVIATIONBAR_STREAMING_WS_MODE=per_symbol plumbed | `OPENDEVIATIONBAR_STREAMING_WS_MODE=per_symbol python3 -c "..."`                                                                                                   | OK: ws_mode reads per_symbol from env var | ✓ PASS             |
| StreamManager accepts ws_mode kwarg at runtime                | `python3 -c "from opendeviationbar._core import StreamManager; StreamManager(['BTCUSDT'], [250], ws_mode='combined')"`                                             | StreamManager created successfully        | ✓ PASS             |
| WsMode unit tests (5 tests)                                   | `cargo nextest run -p opendeviationbar-streaming --features binance-integration -E 'test(ws_mode)                                                                  | test(split_symbols)'`                     | 5 passed, 0 failed | ✓ PASS |
| Stathera continuity 24h+ on bigblack                          | Requires live production run                                                                                                                                       | Not runnable statically                   | ? SKIP             |

### Requirements Coverage

| Requirement | Source Plan   | Description                                                                                                            | Status        | Evidence                                                                                                                                                                                                                              |
| ----------- | ------------- | ---------------------------------------------------------------------------------------------------------------------- | ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| WS-01       | 04-01-PLAN.md | Sidecar uses Binance combined streams with 2x9 hybrid architecture (updated to 2x13) instead of per-symbol connections | ✓ SATISFIED   | start_ws() WsMode::Combined spawns 2 CombinedWebSocketStream tasks with 13+13 volume-ranked symbol split. Combined WS uses Binance /stream?streams= URL format                                                                        |
| WS-02       | 04-01-PLAN.md | Combined stream reconnection uses Antaeus exponential backoff with jitter                                              | ✓ SATISFIED   | combined_ws.rs:build_backoff() uses backon::ExponentialBuilder::with_jitter(). Each of the 2 connections has independent backoff state and child CancellationToken                                                                    |
| WS-03       | 04-02-PLAN.md | All 26 symbols maintain Stathera continuity after migration to combined streams                                        | ? NEEDS HUMAN | Code-level correctness confirmed (mode-agnostic channels, independent connections, 1094 tests pass). Full Stathera continuity validation requires 24h+ production run on bigblack per D-02 decision (tracked as post-deploy activity) |

**Orphaned requirements check:** No Phase 4 requirements in REQUIREMENTS.md beyond WS-01, WS-02, WS-03. All accounted for.

### Anti-Patterns Found

| File       | Line | Pattern | Severity | Impact |
| ---------- | ---- | ------- | -------- | ------ |
| None found | —    | —       | —        | —      |

No stubs, placeholders, TODO markers, or empty implementations found in the 7 modified files. Known stubs sections in both SUMMARY files report "None."

### Human Verification Required

#### 1. Stathera Continuity 24h+ Production Run (WS-03)

**Test:** Deploy sidecar to bigblack with default `OPENDEVIATIONBAR_STREAMING_WS_MODE=combined` (or set explicitly). Run for 24+ hours. After 24h, query ClickHouse for Stathera gaps across all 26 symbols at all thresholds.

**Expected:** Zero Stathera gaps (no rows with `bar[i].first_agg_trade_id != bar[i-1].last_agg_trade_id + 1`) for the 24h window. Any Stathera gaps indicate the combined stream split is causing trade-ID discontinuities not caught by Puck fill.

**Why human:** Cannot simulate live WebSocket behavior statically. The combined stream reconnection edge cases, proactive 23h reconnection, and Puck fill interaction require a real production run to validate.

#### 2. /health Endpoint WS Connection Count Observability (SC-4)

**Test:** After sidecar starts in `combined` mode, `curl http://bigblack:8081/health` and inspect response.

**Expected per ROADMAP SC-4:** Response should confirm "2 WS connections active" (not 18+).

**Why human:** The `/health` endpoint's `connection_count` field is a cumulative counter of CONNECT signals, not a current active count. No field in the current http_server.py response exposes `ws_mode` or active connection count. This observability gap should be assessed: either (a) SC-4 is considered satisfied by the Rust log line "started 2 combined WS connections (group_a=13, group_b=13)" or (b) a `/health` field needs to be added.

**Note:** If the human determines that the log line satisfies SC-4 (rather than requiring the `/health` endpoint), this item can be closed without code changes.

### Gaps Summary

No blocking code gaps. All must_have truths are verified and all artifacts pass existence, substantive, and wiring checks. The two human verification items are:

1. **WS-03 production validation** — explicitly deferred to post-deploy per D-02 decision documented in 04-02-PLAN.md. This is expected and by design.

2. **SC-4 /health observability** — a minor discrepancy between the ROADMAP's stated "observable in sidecar /health endpoint" and the current `/health` response structure. The Rust log line does confirm 2 connections. This is a documentation/observability question, not a functional failure.

The implementation is deployment-ready pending the human verification above.

---

_Verified: 2026-03-24T21:02:22Z_
_Verifier: Claude (gsd-verifier)_
