# Phase 16: Sidecar Aion Startup - Research

**Researched:** 2026-03-27
**Domain:** Sidecar startup path branching for Aion vs Day mode
**Confidence:** HIGH

## Summary

Phase 16 requires the sidecar startup path to branch based on per-symbol `ouroboros_mode`. Currently, `_create_engine()` in `sidecar/__init__.py` unconditionally runs midnight preflight, clean-slate DELETE of today's bars, and `fill_from_rest` from midnight crossover TID. For Aion symbols, this must be replaced with checkpoint-resume: load the last checkpoint TID and resume from there without wiping existing bars.

The Rust side is already prepared: Phase 14 (Plan 02) added `OuroborosMode` enum to the streaming crate with `Day` (default) and `Aion` variants. All 7 call sites in `symbol_task.rs`, `puck.rs`, `mod.rs`, and `trade_dispatch.rs` currently hardcode `OuroborosMode::Day` with explicit `TODO(v7.0 Phase 16)` comments. The `maybe_reset_at_midnight()` function already returns `None` for Aion mode (no midnight reset). Puck gap fill uses `maybe_reset_at_midnight` which is already mode-aware.

**Primary recommendation:** Branch the Python startup path in `_create_engine()` based on per-symbol `ouroboros_mode` from the symbol registry. Wire the actual mode through the Rust `symbol_task` constructor. The cross-midnight bar filter in the main loop must be conditional (skip for Aion symbols).

<phase_requirements>

## Phase Requirements

| ID      | Description                                                                          | Research Support                                                                                                                                                        |
| ------- | ------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| SIDE-01 | Aion symbols skip midnight preflight and clean-slate DELETE on sidecar startup       | Python `_create_engine()` branches: Aion symbols skip `_run_midnight_preflight()` and per-partition DELETE; Day symbols keep existing path                              |
| SIDE-02 | Aion symbols use checkpoint-resume startup (TID-based resume, not genesis recompute) | Checkpoint injection via `_inject_checkpoints()` + `_seed_trade_ids_from_clickhouse()` for Aion symbols; existing checkpoint infrastructure is already built            |
| SIDE-03 | Puck inline gap fill remains wired and operational for Aion symbols                  | Puck is already mode-agnostic for gap detection (TID-based); only `maybe_reset_at_midnight` inside Puck uses mode, and it already handles Aion correctly (returns None) |

</phase_requirements>

## Standard Stack

No new libraries required. This phase modifies existing code paths only.

### Core (existing)

| Library                    | Version   | Purpose                                    | Why Standard                                   |
| -------------------------- | --------- | ------------------------------------------ | ---------------------------------------------- |
| opendeviationbar-streaming | workspace | Rust streaming engine with OuroborosMode   | Already has Aion-aware maybe_reset_at_midnight |
| symbol_registry.py         | n/a       | Python symbol metadata with ouroboros_mode | SSoT for per-symbol mode lookup                |

## Architecture Patterns

### Current Startup Path (Day mode -- `_create_engine()`)

```
1. engine.start_ws()           -- buffer WS trades
2. Skip checkpoint injection   -- fill_from_rest rebuilds from scratch
3. _run_midnight_preflight()   -- get crossover TIDs for all symbols
4. _check_yesterday_integrity()
5. Kill mutations + OPTIMIZE FINAL + DELETE today  -- clean slate
6. _process_parquet_ticks()    -- optional Parquet seed
7. engine.fill_from_rest()     -- REST backfill from midnight crossover
8. _sync_flush_rest_bars()     -- write REST bars to CH
9. engine.start()              -- begin WS processing
```

### Required Aion Startup Path

```
1. engine.start_ws()           -- buffer WS trades (SAME)
2. _inject_checkpoints()       -- restore processor state from checkpoint files
3. _seed_trade_ids_from_clickhouse()  -- seed gap detector from CH last TIDs
4. engine.fill_from_rest()     -- REST backfill from checkpoint/CH seed (NOT midnight)
5. _sync_flush_rest_bars()     -- write REST bars to CH (SAME)
6. engine.start()              -- begin WS processing (SAME)
```

### Key Differences

| Step                      | Day Mode                      | Aion Mode                                  |
| ------------------------- | ----------------------------- | ------------------------------------------ |
| Checkpoint injection      | Skipped (fresh from midnight) | Required (resume from last session)        |
| Midnight preflight        | Runs (get crossover TID)      | Skipped (no day boundary concept)          |
| Yesterday integrity check | Runs                          | Skipped                                    |
| DELETE today              | Runs (clean slate)            | Skipped (preserve existing bars)           |
| Parquet tick processing   | Runs (process today's ticks)  | Skipped (checkpoint has forming bar state) |
| fill_from_rest seed       | Midnight crossover TID        | max(checkpoint_tid, ch_last_tid)           |

### Branching Strategy

The cleanest approach: partition `config.symbols` into `day_symbols` and `aion_symbols` at the top of `_create_engine()`, then conditionally execute Day-only steps for Day symbols only. Since the Rust engine handles all symbols in a single `StreamManager`, the branching is in the Python startup orchestration only.

**Critical insight:** `fill_from_rest` in the Rust engine already handles per-symbol seed TIDs via `gap_detector_seeds`. For Day symbols, seeds come from midnight preflight; for Aion symbols, seeds come from checkpoints + CH. The Rust `fill_from_rest` does not need to know about Aion -- it just uses whatever seed TID was provided.

### Rust-Side Wiring Pattern

The `symbol_task` function receives an `OuroborosMode` parameter that gets threaded through to `process_trade_through_processors` and `puck_fill`. Currently hardcoded as `OuroborosMode::Day`. The wiring needs:

1. `LiveBarEngine::new()` / `StreamManager` constructor receives per-symbol mode config
2. Stored in a `HashMap<String, OuroborosMode>` on the engine
3. Passed to `symbol_task()` as a new parameter
4. `symbol_task` threads it to all `process_trade_through_processors` and `puck_fill` call sites
5. Python `StreamManager` constructor exposed via PyO3 (or new method like `set_symbol_mode()`)

### Python Main Loop Cross-Midnight Filter

Lines 1168-1183 in `sidecar/__init__.py` discard bars where `open_day != close_day`. This is a Day-mode safety filter. For Aion symbols, cross-midnight bars are valid and expected. This filter must be conditional on mode.

```python
# Current (unconditional):
if (open_us // 86_400_000_000) != (close_us // 86_400_000_000):
    logger.warning("cross-midnight bar discarded...")
    continue

# Required (conditional):
if symbol_modes[symbol] == "day" and (open_us // 86_400_000_000) != (close_us // 86_400_000_000):
    logger.warning("cross-midnight bar discarded...")
    continue
```

### sync_flush Cross-Midnight Filter

`_sync_flush_rest_bars()` in `midnight.py` (lines 504-515) also has a cross-midnight filter that must be conditioned on mode.

### Anti-Patterns to Avoid

- **Global mode switch:** Do NOT use `get_operational_ouroboros_mode()` which returns a single global mode. The sidecar handles mixed Day+Aion symbols simultaneously (e.g., forex=Day, crypto=Aion). Mode must be per-symbol.
- **Separate engines for Day/Aion:** The `StreamManager` already handles all symbols -- splitting would duplicate WS connections and add complexity.
- **Skipping fill_from_rest for Aion:** Even Aion symbols need REST backfill from their last checkpoint to catch up to live. The difference is the seed TID source, not whether fill happens.

## Don't Hand-Roll

| Problem                | Don't Build                | Use Instead                                             | Why                                                               |
| ---------------------- | -------------------------- | ------------------------------------------------------- | ----------------------------------------------------------------- |
| Per-symbol mode lookup | Custom config parsing      | `get_symbol_entries()` from symbol_registry             | Already has `ouroboros_mode` field (Phase 13 REG-01/REG-02)       |
| Checkpoint resume      | New checkpoint format      | Existing `_inject_checkpoints()` + `_load_checkpoint()` | Already built and tested for watchdog restarts                    |
| Gap detection in Aion  | Mode-specific gap detector | Existing `TradeIdGapDetector` + Puck                    | Already TID-based, mode-agnostic                                  |
| Seed TID resolution    | Custom seed logic          | Existing `max(checkpoint_tid, ch_seed_tid)` pattern     | Already in `symbol_task.rs` line 72 and `fill_from_rest` line 420 |

## Common Pitfalls

### Pitfall 1: Wiping Aion bars on restart

**What goes wrong:** If Aion symbols go through the Day clean-slate path, all existing bars for today (which may span many hours) are deleted and recomputed only from midnight, losing the continuous chain.
**Why it happens:** The `needs_fill` flag (line 230) is currently set for ALL engines with `fill_from_rest`, not per-symbol.
**How to avoid:** Branch the startup orchestration per-symbol based on mode BEFORE the DELETE step.
**Warning signs:** Bars_written count drops dramatically after restart for Aion symbols.

### Pitfall 2: Mixed Day+Aion symbol handling in DELETE loop

**What goes wrong:** The DELETE loop (lines 330-351) iterates all registered symbols. If Aion symbols are in the registry, their bars get deleted even though they shouldn't be.
**Why it happens:** The loop uses `get_registered_symbols(enabled_only=True)` without filtering by mode.
**How to avoid:** Filter the DELETE loop to Day-mode symbols only.
**Warning signs:** Aion bars disappear from CH after every restart.

### Pitfall 3: Cross-midnight filter dropping valid Aion bars

**What goes wrong:** Aion bars that span UTC midnight are silently discarded by the main loop filter.
**Why it happens:** The filter at line 1172 is unconditional.
**How to avoid:** Make the filter conditional on `symbol_modes[symbol]`.
**Warning signs:** Missing bars around midnight for Aion symbols, Stathera gaps at day boundaries.

### Pitfall 4: Puck using Day mode during gap fill for Aion symbols

**What goes wrong:** `puck_fill` in `puck.rs` line 84 hardcodes `OuroborosMode::Day`. During Puck gap fill for an Aion symbol, trades crossing midnight would trigger an orphan reset, corrupting the continuous bar chain.
**Why it happens:** Phase 14 left all call sites as `OuroborosMode::Day` placeholders.
**How to avoid:** Thread per-symbol mode to `puck_fill()` via the same `HashMap<String, OuroborosMode>`.
**Warning signs:** Unexpected orphan bars appearing in Aion data around midnight.

### Pitfall 5: Yesterday integrity check failing for Aion symbols

**What goes wrong:** `_check_yesterday_integrity()` checks for orphan bars and Stathera gaps per-day. Aion symbols don't have orphan bars (no midnight reset), so they'll always be flagged as broken.
**Why it happens:** The check assumes Day-mode behavior (orphan at midnight boundary).
**How to avoid:** Skip yesterday integrity check for Aion symbols.
**Warning signs:** False positive kintsugi repair signals for Aion symbols.

## Code Examples

### Per-symbol mode lookup in Python startup

```python
# Source: symbol_registry.py (already implemented in Phase 13)
from opendeviationbar.symbol_registry import get_symbol_entries
from opendeviationbar.ouroboros import OuroborosMode

entries = get_symbol_entries()
symbol_modes = {
    sym: entry.ouroboros_mode
    for sym, entry in entries.items()
    if sym in config.symbols
}
aion_symbols = [s for s, m in symbol_modes.items() if m == OuroborosMode.AION]
day_symbols = [s for s, m in symbol_modes.items() if m == OuroborosMode.DAY]
```

### Branching \_create_engine startup

```python
# Aion path: inject checkpoints + seed from CH
if aion_symbols:
    loaded, loaded_pairs = _inject_checkpoints(
        engine, aion_symbols, config.thresholds, trace_id,
        gap_fill_results=gap_fill_results, valid_pairs=valid_pairs,
    )
    _seed_trade_ids_from_clickhouse(engine, aion_symbols, config.thresholds)

# Day path: midnight preflight + clean-slate DELETE
if day_symbols:
    preflight_verified = _run_midnight_preflight(engine, day_symbols, config.thresholds)
    # ... check yesterday integrity for day_symbols only ...
    # ... DELETE today for day_symbols only ...
    # ... process parquet ticks for day_symbols only ...
```

### Rust-side mode wiring in symbol_task

```rust
// In symbol_task signature (new parameter):
pub(crate) async fn symbol_task(
    symbol: String,
    ouroboros_mode: OuroborosMode,  // NEW: per-symbol mode
    // ... rest of params ...
) {
    // Replace all hardcoded OuroborosMode::Day with ouroboros_mode
    process_trade_through_processors(
        trade, &mut processors, &mut last_days,
        &bar_buffer, &metrics, &symbol_arc, &symbol,
        &forming_tx_map, &mut committed_floors,
        &mut last_forming_update,
        ouroboros_mode,  // was: OuroborosMode::Day
    );
}
```

### PyO3 mode passing

```python
# Option A: New method on StreamManager
engine.set_symbol_ouroboros_modes({"BTCUSDT": "aion", "EURUSD": "day"})

# Option B: Constructor parameter
engine = StreamManager(
    symbols, thresholds, include_microstructure,
    # ... existing params ...
    symbol_modes={"BTCUSDT": "aion", "EURUSD": "day"},
)
```

## State of the Art

| Old Approach                      | Current Approach                     | When Changed       | Impact                               |
| --------------------------------- | ------------------------------------ | ------------------ | ------------------------------------ |
| Global ouroboros_mode             | Per-symbol mode from symbols.toml    | Phase 13 (REG-01)  | Registry already has per-symbol mode |
| Hardcoded Day in Rust             | OuroborosMode enum with Aion variant | Phase 14 (Plan 02) | Rust code ready, needs wiring        |
| Unconditional clean-slate startup | Must branch per mode                 | This phase         | Core change                          |

## Validation Architecture

### Test Framework

| Property           | Value                                                   |
| ------------------ | ------------------------------------------------------- |
| Framework          | pytest + cargo nextest                                  |
| Config file        | `pyproject.toml` [tool.pytest] + `.cargo/config.toml`   |
| Quick run command  | `cargo nextest run -p opendeviationbar-streaming --lib` |
| Full suite command | `mise run check-full`                                   |

### Phase Requirements to Test Map

| Req ID  | Behavior                                      | Test Type | Automated Command                                                       | File Exists?                                   |
| ------- | --------------------------------------------- | --------- | ----------------------------------------------------------------------- | ---------------------------------------------- |
| SIDE-01 | Aion symbols skip midnight preflight + DELETE | unit      | `pytest tests/test_sidecar.py -x -k aion_startup`                       | Wave 0                                         |
| SIDE-02 | Aion checkpoint-resume startup                | unit      | `pytest tests/test_sidecar.py -x -k aion_checkpoint`                    | Wave 0                                         |
| SIDE-03 | Puck operational for Aion                     | unit      | `cargo nextest run -p opendeviationbar-streaming --lib -E 'test(puck)'` | Existing (puck tests exist, need Aion variant) |

### Sampling Rate

- **Per task commit:** `cargo nextest run -p opendeviationbar-streaming --lib && pytest tests/test_sidecar.py -x`
- **Per wave merge:** `mise run check-full`
- **Phase gate:** Full suite green before `/gsd:verify-work`

### Wave 0 Gaps

- [ ] `tests/test_sidecar.py::test_aion_startup_skips_midnight_preflight` -- covers SIDE-01
- [ ] `tests/test_sidecar.py::test_aion_startup_uses_checkpoint_resume` -- covers SIDE-02
- [ ] Rust test: Aion mode puck_fill does not produce orphan bars -- covers SIDE-03

## Detailed Code Inventory: All Changes Required

### Rust Changes (crates/opendeviationbar-streaming)

| File                         | Change                                                                                       | Lines                                          |
| ---------------------------- | -------------------------------------------------------------------------------------------- | ---------------------------------------------- |
| `live_engine/mod.rs`         | Add `symbol_modes: HashMap<String, OuroborosMode>` to `LiveBarEngine` and `LiveEngineConfig` | Config struct + constructor                    |
| `live_engine/mod.rs`         | Pass `ouroboros_mode` to `symbol_task()` spawn call in `start()`                             | ~line 230                                      |
| `live_engine/mod.rs`         | Replace `OuroborosMode::Day` in `fill_from_rest` REST chunk loop                             | line 561                                       |
| `live_engine/symbol_task.rs` | Add `ouroboros_mode: OuroborosMode` parameter                                                | Signature + 3 call sites (lines 219, 278, 434) |
| `live_engine/puck.rs`        | Add `ouroboros_mode: OuroborosMode` parameter, pass to `maybe_reset_at_midnight`             | line 84                                        |

### PyO3 Binding Changes (src/)

| File                             | Change                                                                                            |
| -------------------------------- | ------------------------------------------------------------------------------------------------- |
| `src/stream_manager_bindings.rs` | Accept `symbol_modes: Option<HashMap<String, String>>` in constructor, convert to `OuroborosMode` |

### Python Changes (python/opendeviationbar/sidecar/)

| File          | Change                                                                             |
| ------------- | ---------------------------------------------------------------------------------- |
| `__init__.py` | Partition symbols into day/aion in `_create_engine()`                              |
| `__init__.py` | Conditional startup: checkpoints for Aion, midnight preflight for Day              |
| `__init__.py` | Conditional DELETE: only Day symbols                                               |
| `__init__.py` | Conditional cross-midnight filter in main loop                                     |
| `__init__.py` | Pass `symbol_modes` dict to `StreamManager` constructor                            |
| `midnight.py` | `_run_midnight_preflight()`: only accept Day symbols (already filters non-Binance) |
| `midnight.py` | `_check_yesterday_integrity()`: only check Day symbols                             |
| `midnight.py` | `_sync_flush_rest_bars()`: conditional cross-midnight filter                       |
| `recovery.py` | `_restart_engine_with_recovery()`: must preserve per-symbol mode context           |

## Open Questions

1. **PyO3 API shape for passing symbol modes**
   - What we know: StreamManager constructor needs per-symbol mode mapping
   - What's unclear: Whether to add it to constructor vs. a `set_symbol_modes()` method called before `start()`
   - Recommendation: Constructor parameter -- simplest, and mode is immutable after creation. Use `Option<HashMap<String, String>>` with None defaulting all to Day.

2. **fill_from_rest behavior for Aion symbols**
   - What we know: fill_from_rest creates fresh processors if no checkpoint. For Aion, we want checkpoint-restored processors.
   - What's unclear: Whether `_inject_checkpoints` on the Python side properly feeds into `fill_from_rest`'s processor creation
   - Recommendation: The existing `initial_checkpoints` mechanism in `LiveBarEngine` already handles this -- `_inject_checkpoints` calls `engine.set_checkpoint()` which populates `initial_checkpoints`. `fill_from_rest` reads from there. This should work as-is for Aion.

## Sources

### Primary (HIGH confidence)

- Codebase inspection of `sidecar/__init__.py` (1317 lines) -- full startup path
- Codebase inspection of `sidecar/midnight.py` (896 lines) -- preflight, fill_from_rest, sync_flush
- Codebase inspection of `symbol_task.rs`, `trade_dispatch.rs`, `puck.rs` -- Rust-side OuroborosMode wiring
- Phase 14 summaries -- confirmed OuroborosMode enum + TODO markers

### Secondary (MEDIUM confidence)

- Phase 14-02 SUMMARY confirms 7 call sites with `OuroborosMode::Day` placeholder

## Metadata

**Confidence breakdown:**

- Standard stack: HIGH - no new dependencies, all existing code
- Architecture: HIGH - startup path and Rust wiring fully traced through codebase
- Pitfalls: HIGH - identified from reading actual code paths and understanding the Day-mode invariants that must be relaxed for Aion

**Research date:** 2026-03-27
**Valid until:** 2026-04-27 (stable internal codebase, no external dependencies)
