# Rust Processor State Invariant Guard Investigation

**Date**: 2026-03-21
**Perspective**: Can `OpenDeviationBarProcessor` enter an invalid state that produces wrong bars?

---

## TGI-1: Mutable State Field Catalog

### All 16 mutable fields in `OpenDeviationBarProcessor`

| #   | Field                          | Type                            | Updated When                                          | Reset at Ouroboros?                                  |
| --- | ------------------------------ | ------------------------------- | ----------------------------------------------------- | ---------------------------------------------------- |
| 1   | `threshold_decimal_bps`        | `u32`                           | Constructor only (immutable after creation)           | No (preserved)                                       |
| 2   | `threshold_ratio`              | `i64`                           | Constructor only (derived from #1)                    | No (preserved)                                       |
| 3   | `current_bar_state`            | `Option<OpenDeviationBarState>` | Every trade, bar completion, ouroboros                | Yes (taken, returned as orphan)                      |
| 4   | `price_window`                 | `PriceWindow`                   | Every trade (push)                                    | Yes (reset to new)                                   |
| 5   | `last_trade_id`                | `Option<i64>`                   | Every trade                                           | Yes (reset to None)                                  |
| 6   | `last_completed_bar_tid`       | `Option<i64>`                   | Bar completion + orphan emission only                 | **Partially** (updated if orphan, NOT reset to None) |
| 7   | `last_timestamp_us`            | `i64`                           | Every trade                                           | Yes (reset to 0)                                     |
| 8   | `anomaly_summary`              | `AnomalySummary`                | Gap detection in batch path                           | No (preserved across boundaries)                     |
| 9   | `resumed_from_checkpoint`      | `bool`                          | `from_checkpoint()` sets true; batch path consumes it | Yes (reset to false)                                 |
| 10  | `prevent_same_timestamp_close` | `bool`                          | Constructor only (immutable after creation)           | No (preserved)                                       |
| 11  | `defer_open`                   | `bool`                          | Bar completion sets true; next trade resets false     | Yes (reset to false)                                 |
| 12  | `trade_history`                | `Option<TradeHistory>`          | Every trade (push), bar open/close, config methods    | Partially (bar boundaries reset, trades preserved)   |
| 13  | `inter_bar_config`             | `Option<InterBarConfig>`        | Config methods only                                   | No (preserved)                                       |
| 14  | `include_intra_bar_features`   | `bool`                          | Config methods only                                   | No (preserved)                                       |
| 15  | `intra_bar_config`             | `IntraBarConfig`                | Config methods only                                   | No (preserved)                                       |
| 16  | `max_gap_us`                   | `i64`                           | Config method only                                    | No (preserved)                                       |

---

## TGI-2: Invariant Analysis — What Can Go Wrong?

### Finding 1: Batch Path `defer_open` Desync (LOW severity for production)

**Location**: `processor.rs:649-860` (`process_agg_trade_records_with_options`)

The batch path creates a **local** `let mut defer_open = false;` (line 691) that shadows `self.defer_open`. When the batch completes, the local `defer_open` is NOT synced back to `self.defer_open`.

**Impact**: If the last trade in a batch triggers a breach:

- Local `defer_open = true`, `current_bar = None` (bar was taken)
- `self.current_bar_state = None` (correct via line 856)
- But `self.defer_open` remains `false` (WRONG — should be `true`)
- Subsequent `create_checkpoint()` would serialize `defer_open = false`
- Subsequent `process_single_trade()` would NOT defer open — the first trade would be treated as "first trade, initialize new bar" via the `None` arm at line 469, which happens to produce the correct result anyway since `current_bar_state` is `None`.

**Mitigating factor**: When `current_bar_state = None` and `defer_open = false`, the next `process_single_trade()` takes the `None =>` branch and opens a new bar — which is identical behavior to `defer_open = true` when `current_bar_state = None`. So the behavior is accidentally correct. But the invariant is still violated, and `create_checkpoint()` would serialize the wrong value.

**Production impact**: LOW. The batch path is NOT used in the streaming sidecar (which uses `process_single_trade` exclusively). The batch path is used for backfill/kintsugi where `create_checkpoint()` is called after processing. However, since the next batch call resets state anyway (`self.current_bar_state = None` at line 686), this only matters if someone interleaves batch and single-trade calls, which doesn't happen in practice.

**Recommended fix**: Add `self.defer_open = defer_open;` before `Ok(bars)` at line 859.

### Finding 2: `last_completed_bar_tid` Survives Ouroboros (CORRECT by design)

`reset_at_ouroboros()` updates `last_completed_bar_tid` if there's an orphan but does NOT reset it to `None`. This is intentional — the comment at line 1143-1144 explicitly documents this: "Do NOT reset last_completed_bar_tid to None — it persists across day boundaries."

This is correct: `committed_floors` needs the floor to persist so it can suppress already-written bars even after midnight reset.

### Finding 3: `seed_trade_id()` Takes MAX — Ordering-Sensitive (KNOWN risk)

`StreamManager::seed_trade_id()` (stream_manager.rs:182-188) and `OdbEngine::seed_trade_id()` (odb_engine.rs:66-74) both use MAX semantics: `if last_agg_trade_id > *entry { *entry = last_agg_trade_id; }`.

This means the ordering of seed calls matters:

1. If CH seed runs BEFORE `delete_today_bars`, the stale high-water mark from today's bars inflates the seed
2. Midnight preflight's crossover TID override is silently ignored if seed is already higher

This is the **exact bug** documented in `feedback-seed-ordering.md` in MEMORY.md — the startup sequence ordering invariant (CLAUDE.md section 6). The fix is procedural (enforce ordering in Python `_create_engine()`), not structural in Rust.

### Finding 4: `puck_fill` Does Not Check `committed_floors` (ACCEPTABLE)

`puck_fill()` (live_engine.rs:1191) emits bars without checking `committed_floors`. This is acceptable because:

- `puck_fill` fills gaps in trade IDs the processor hasn't seen
- Bars from gap-fill should always be genuinely new
- The dedup safety net exists in Python/ClickHouse (RMT(computed_at) INSERT-only dedup, L5)

### Finding 5: `fill_from_rest` Does Not Update `committed_floors` (ACCEPTABLE)

`fill_from_rest()` (live_engine.rs:361) runs BEFORE the symbol_task loop where `committed_floors` is initialized. The processors created in `fill_from_rest` are stored in `pre_created_processors` and reused in `start()`. The `committed_floors` initialization (line 1387-1396) reads `processor.last_completed_bar_tid()` from these processors, so the floor correctly reflects what `fill_from_rest` produced. No issue.

### Finding 6: Checkpoint Does Not Preserve `trade_history` or Feature Config (KNOWN)

`from_checkpoint()` sets:

- `trade_history: None`
- `inter_bar_config: None`
- `include_intra_bar_features: false`

The code explicitly documents this: "Issue #59: Must be re-enabled after restore". Callers (e.g., `fill_from_rest` at line 433-438) properly re-enable these after restoring from checkpoint. Not a bug, but a footgun — if someone forgets to re-enable after `from_checkpoint()`, inter/intra-bar features silently become `None`.

### Finding 7: No Guard Against Backward Trade IDs (PARTIAL)

`process_single_trade()` unconditionally sets `self.last_trade_id = Some(trade.agg_trade_id)` (line 442). There is no check that the new trade ID is greater than the previous one. The batch path validates ordering via `validate_trade_ordering()`, but the single-trade path does not.

**Impact**: If trades arrive out of order via `process_single_trade()`:

- `last_trade_id` goes backward
- `last_completed_bar_tid` could go backward if a bar completes
- `committed_floors` could be set to a lower value
- Gap detection would produce incorrect results

**Mitigating factor**: In the streaming path, the WS feed delivers trades in order, and `puck_fill` fetches from REST in agg_trade_id order. The main defense is at the data source level. But there's no Rust-level guard.

---

## Summary of Findings

| #   | Finding                                          | Severity                            | Action                        |
| --- | ------------------------------------------------ | ----------------------------------- | ----------------------------- |
| 1   | Batch `defer_open` desync                        | LOW (accidentally correct behavior) | Fix: sync local to self       |
| 2   | `last_completed_bar_tid` survives ouroboros      | N/A (correct by design)             | None                          |
| 3   | `seed_trade_id` MAX semantics ordering-sensitive | HIGH (known, documented)            | Procedural (startup ordering) |
| 4   | `puck_fill` no committed_floors check            | ACCEPTABLE                          | Dedup at L5                   |
| 5   | `fill_from_rest` doesn't update committed_floors | N/A (floors init'd after)           | None                          |
| 6   | Checkpoint drops feature config                  | KNOWN (documented)                  | Footgun warning only          |
| 7   | No backward trade ID guard in single-trade path  | MEDIUM                              | Consider debug_assert or warn |

### Overall Assessment

The processor's mutable state is well-managed. The `last_completed_bar_tid` fix (v1.4) was the most critical invariant violation, and it has been properly addressed. The remaining findings are either low-severity (batch defer_open desync), procedural (seed ordering), or defensive (backward trade ID guard). The architecture is sound — the main risk surface is at the Python orchestration layer (startup sequence ordering), not in Rust state management.
