# Ring Buffer Capacity Analysis

**Requirement**: AUDT-01
**Date**: 2026-03-19
**Author**: Plan 02-02

---

## Summary

The current ring buffer capacity of **10,000 bars** (`OPENDEVIATIONBAR_MAX_PENDING_BARS`, default) is
**sufficient for the current 2-symbol deployment** but is **INSUFFICIENT for a full 18-symbol deployment**.
For Phase 4 (parallel fill), the recommended capacity is **100,000** bars.

---

## 1. Current Configuration

| Setting          | Value                                                | Source                 |
| ---------------- | ---------------------------------------------------- | ---------------------- |
| Env var          | `OPENDEVIATIONBAR_MAX_PENDING_BARS`                  | `live_engine.rs:136`   |
| Default capacity | **10,000 bars**                                      | `live_engine.rs:139`   |
| Buffer type      | `ConcurrentRingBuffer<CompletedBar>`                 | `ring_buffer.rs:142`   |
| Drop policy      | **Drop oldest** when full (`push()` returns `false`) | `ring_buffer.rs:60-68` |
| Memory footprint | ~5 MB at 10K capacity (500 bytes/bar est.)           | —                      |

### Drop-on-Full Semantics

`ConcurrentRingBuffer` uses a `parking_lot::Mutex`-guarded `RingBuffer<T>`. When the buffer is full
and `push()` is called, the **oldest bar is silently dropped** and the new bar takes its slot. The
engine logs a warning (`tracing::warn!("ring buffer full, old bar dropped")`) and increments
`dropped_bars` + `backpressure_events` metrics. There is **no backpressure to the producer** — the
trade processing hot path is never blocked.

This means: if Python is not consuming bars fast enough, bars are lost permanently.

---

## 2. Critical Insight: Python Does NOT Consume During fill_from_rest

`fill_from_rest()` is called synchronously via PyO3 (`engine.fill_from_rest()` in `sidecar.py:852`).
While Rust is executing `fill_from_rest`, the Python sidecar main loop is **blocked waiting** — it
cannot call `collect_bars()` or drain the ring buffer.

This means the ring buffer must hold **all bars produced during the entire fill** without being drained.

**Code evidence** (`live_engine.rs:561-582`):

```rust
match processor.process_single_trade(trade) {
    Ok(Some(bar)) => {
        let _ = self.bar_buffer.push(completed);  // Accumulates; never drained during fill
        total_bars += 1;
    }
    ...
}
```

---

## 3. Worst-Case Bar Production Rate

### Inputs

| Parameter                   | Value                               | Source                 |
| --------------------------- | ----------------------------------- | ---------------------- |
| Symbols (Phase 4 target)    | 18 (TIER1_SYMBOLS)                  | `constants.py:230-248` |
| Thresholds                  | 4 (250, 500, 750, 1000 dbps)        | Production config      |
| Processor instances         | 72 (18 × 4)                         | Derived                |
| Chunk size (fill_from_rest) | 50,000 trades                       | `live_engine.rs:462`   |
| Chunks per symbol           | Varies; today's gap = minutes→hours | Observed               |

### Bar Production Estimates

The number of bars produced per chunk depends heavily on the threshold and symbol volatility.
Lower thresholds produce more bars from the same trades:

| Threshold         | Approx trades/bar (BTCUSDT) | Bars per 50K-trade chunk |
| ----------------- | --------------------------- | ------------------------ |
| 250 dbps (0.25%)  | ~50–200                     | 250–1,000                |
| 500 dbps (0.50%)  | ~200–800                    | 60–250                   |
| 750 dbps (0.75%)  | ~500–2,000                  | 25–100                   |
| 1000 dbps (1.00%) | ~800–3,000                  | 17–63                    |

**Worst-case (most volatile period, lowest threshold):**

- BTCUSDT at 250 dbps: ~1,000 bars per 50K-trade chunk

### Fill Duration (Today's Gap)

`fill_from_rest` fills from the last committed ClickHouse trade ID to the current WS position.
With the current 2-symbol production setup:

- **Observed**: ~5 minutes total fill time for 2 symbols
- **18-symbol extrapolation**: Sequential fill → ~45 minutes
- Trades per symbol in a 5-minute gap on BTCUSDT: ~5,000–15,000 trades (1–3 chunks)
- Bars per symbol in a 5-minute gap at 250 dbps: ~50–300 bars

### After a Full-Day Restart (Cold Start)

If the sidecar has been stopped for a full day (e.g., overnight maintenance):

| Scenario                                       | Trades to process     | Bars produced (est.) | Thresholds |
| ---------------------------------------------- | --------------------- | -------------------- | ---------- |
| 1 symbol, 1 threshold                          | ~1M–2M (busy day)     | 2,000–20,000         | 1          |
| 1 symbol, 4 thresholds                         | ~1M–2M                | 5,000–30,000         | 4          |
| 18 symbols, 4 thresholds (sequential)          | 18M–36M               | 90,000–540,000       | 4          |
| 18 symbols, 4 thresholds (parallel, 4 workers) | 18M–36M (in parallel) | Same total           | 4          |

**BTCUSDT-specific** (highest bar rate, ~8M trades/day):

- At 250 dbps: ~40,000–80,000 bars per day
- Sequential fill of BTCUSDT alone could overflow 10K ring buffer within a single chunk

---

## 4. Current Production Safety Analysis

### Today's Deployment (2 symbols, 4 thresholds)

| Symbol             | Est. bars in 5-min gap | Est. bars in 1-hour gap | Full day restart |
| ------------------ | ---------------------- | ----------------------- | ---------------- |
| BTCUSDT @250       | ~100                   | ~1,200                  | ~60,000          |
| BTCUSDT @500       | ~30                    | ~360                    | ~18,000          |
| BTCUSDT @750       | ~15                    | ~180                    | ~9,000           |
| BTCUSDT @1000      | ~10                    | ~120                    | ~6,000           |
| **BTCUSDT total**  | **~155**               | **~1,860**              | **~93,000**      |
| ETHUSDT (similar)  | ~120                   | ~1,440                  | ~72,000          |
| **2-symbol total** | **~275**               | **~3,300**              | **~165,000**     |

**Finding**: Normal restarts (short gap) are safe at 10K. A full-day restart with 2 symbols already
exceeds 10K by ~16×. This explains why `dropped_bars` metrics are non-zero after any extended outage.

### Phase 4 Target (18 symbols, 4 thresholds, sequential fill)

Worst case (each symbol has several hours of gap to fill):

```
18 symbols × ~1,860 bars/symbol (1-hour gap) = 33,480 bars
```

This **exceeds 10K by 3.3×** even for modest gaps. For a full-day restart:

```
18 symbols × ~93,000 bars/symbol = 1,674,000 bars (theoretical max)
```

Capacity needed: **~1.7M bars** for worst-case full-day restart of all 18 symbols.

---

## 5. Practical Risk Assessment

### When Do Drops Actually Matter?

Bar drops during `fill_from_rest` create **Stathera gaps**: bars are produced in Rust, emitted to
the ring buffer, dropped (ring full), and then written to ClickHouse with trade-ID discontinuities.
Kintsugi will detect and repair these gaps, but this creates unnecessary repair load.

### Normal Operation (Current 2-Symbol Setup)

- **Short restarts (<30 min)**: ~300–1,000 bars total → **safe at 10K**
- **Extended outages (>2 hours)**: potential overflow → **moderate risk**
- **Full-day restart**: ~165K bars → **certain overflow, kintsugi repairs**

### Phase 4 (18-Symbol Setup)

- **Any restart >10 minutes**: likely overflow → **high risk**
- Bars are dropped silently; kintsugi repairs the Stathera gaps but adds load

---

## 6. Recommendation

### For Phase 4: Increase to 100,000

Set `OPENDEVIATIONBAR_MAX_PENDING_BARS=100000` in `deploy/opendeviationbar.env`.

**Rationale:**

- Covers normal restarts for all 18 symbols (short gaps: ~33K bars)
- Reasonable memory footprint: ~50 MB at 100K capacity (vs. ~5 MB today)
- Does not require code changes — env var already supported since Issue #96

**Not sufficient for**: Full-day cold restart of all 18 symbols (needs ~1.7M capacity).
For full-day restarts, the recommended approach is to use `kintsugi:catchup` separately, not
`fill_from_rest`.

### Longer Term: Bounded Channel with Backpressure

The drop-on-full design is intentional for the real-time WS hot path — dropping old bars keeps
memory bounded and prevents backpressure from stalling the symbol tasks. However, during
`fill_from_rest`, a different strategy is appropriate:

**Option A**: Dedicated `fill_from_rest` output channel (bounded, blocks producer) — zero drops
guaranteed, but adds complexity.

**Option B**: Python async drain during `fill_from_rest` via a background thread — would require
PyO3 GIL management changes.

**Option C** (recommended near-term): Use `OPENDEVIATIONBAR_MAX_PENDING_BARS=100000` and accept
occasional drops on full-day restarts (kintsugi repairs them). Revisit when 18-symbol deployment
is active and real metrics are available from TELE-05.

### Phase 4 Prerequisite Action

Before enabling 18-symbol streaming (Phase 4), add to `deploy/opendeviationbar.env`:

```env
OPENDEVIATIONBAR_MAX_PENDING_BARS=100000
```

Flag in `deploy/CLAUDE.md` under "Phase 4 Prerequisites".

---

## 7. Validation via TELE-05

Plan 02-01 added `bars_produced` telemetry to `fill_from_rest.symbol_summary` log events (TELE-05).
After the next deployment, search journal logs for real production numbers:

```bash
journalctl -u opendeviationbar-sidecar -o json \
  | jq 'select(.MESSAGE | contains("fill_from_rest.symbol_summary"))
        | {symbol: .SYMBOL, bars_produced: .BARS_PRODUCED, trade_count: .TRADE_COUNT}'
```

These real numbers will validate or refine the estimates in Section 3.

---

## References

| File                                                   | Lines     | Content                                                  |
| ------------------------------------------------------ | --------- | -------------------------------------------------------- |
| `crates/opendeviationbar-streaming/src/live_engine.rs` | 132-140   | `get_bar_channel_capacity()`, default 10K                |
| `crates/opendeviationbar-streaming/src/live_engine.rs` | 236       | `ConcurrentRingBuffer::new(config.bar_channel_capacity)` |
| `crates/opendeviationbar-streaming/src/live_engine.rs` | 553-582   | `fill_from_rest` bar accumulation                        |
| `crates/opendeviationbar-streaming/src/live_engine.rs` | 1038-1044 | WS hot path drop logging                                 |
| `crates/opendeviationbar-streaming/src/ring_buffer.rs` | 58-85     | Drop-on-full push semantics                              |
| `python/opendeviationbar/sidecar.py`                   | 850-858   | Python-side `fill_from_rest()` call                      |
| `python/opendeviationbar/constants.py`                 | 230-248   | 18 TIER1_SYMBOLS                                         |
