# CLAUDE.md - Project Memory

**opendeviationbar-py**: Rust workspace with Python bindings via PyO3/maturin. Publishes to PyPI (`opendeviationbar`) and crates.io (`opendeviationbar-core`, `opendeviationbar-providers`, `opendeviationbar-streaming`, `opendeviationbar-client`, `opendeviationbar-hurst`).

> **Naming**: The Open Deviation Bar is a genuinely novel bar type. It uses **open-anchored deviation thresholds**: breach when `price >= open * (1 + threshold)` or `price <= open * (1 - threshold)`. This is distinct from classic Nicolellis-style bars (which use a dynamic `High - Low` distance). The name precisely describes the mechanism: open-anchored, bidirectional, percentage-threshold.
>
> **Full explanation + bit-exact oracle proof**: [docs/OPEN-DEVIATION-BAR.md](/docs/OPEN-DEVIATION-BAR.md)

---

## Quick Reference

| Task                         | Entry Point                                                                                                                                                         | Details                                                                         |
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- |
| Bar algorithm & oracle proof | [docs/OPEN-DEVIATION-BAR.md](/docs/OPEN-DEVIATION-BAR.md)                                                                                                           | Layman + 866K-bar bit-exact ClickHouse examples                                 |
| Generate open deviation bars | `get_open_deviation_bars()`                                                                                                                                         | [Python API](#python-api)                                                       |
| Real-time streaming          | `run_sidecar()` / `scripts/streaming_sidecar.py`                                                                                                                    | [ADR](/docs/adr/2026-01-31-realtime-streaming-api.md)                           |
| Understand architecture      | [docs/ARCHITECTURE.md](/docs/ARCHITECTURE.md)                                                                                                                       | 10-crate workspace                                                              |
| Work with Rust crates        | [crates/CLAUDE.md](/crates/CLAUDE.md)                                                                                                                               | Core algorithm, microstructure, inter-bar features                              |
| Work with Python layer       | [python/opendeviationbar/CLAUDE.md](/python/opendeviationbar/CLAUDE.md)                                                                                             | API, caching, validation, symbol registry                                       |
| ClickHouse cache ops         | [python/opendeviationbar/clickhouse/CLAUDE.md](/python/opendeviationbar/clickhouse/CLAUDE.md)                                                                       | Schema, population, remote setup, dedup hardening                               |
| Plugin system                | [python/opendeviationbar/plugins/CLAUDE.md](/python/opendeviationbar/plugins/CLAUDE.md)                                                                             | FeatureProvider protocol, entry-point discovery                                 |
| Operations & scripts         | [scripts/CLAUDE.md](/scripts/CLAUDE.md)                                                                                                                             | Kintsugi, cache population, parallelism, monitoring                             |
| Exness forex seeder          | `scripts/exness_tick_cache_seeder.py`                                                                                                                               | [scripts/CLAUDE.md](/scripts/CLAUDE.md#exness-forex-tick-cache-seeder-phase-02) |
| Deploy & monitoring          | [deploy/CLAUDE.md](/deploy/CLAUDE.md)                                                                                                                               | Env files, monit watchdog, defense-in-depth                                     |
| Release workflow             | [docs/development/RELEASE.md](/docs/development/RELEASE.md)                                                                                                         | Zig cross-compile, mise tasks                                                   |
| Release + Deploy             | [.claude/skills/odb-ship/SKILL.md](/.claude/skills/odb-ship/SKILL.md)                                                                                               | Unified release-publish-deploy pipeline                                         |
| Performance monitoring       | [docs/development/PERFORMANCE.md](/docs/development/PERFORMANCE.md)                                                                                                 | Benchmarks, metrics                                                             |
| Performance optimization     | [.claude/pgo-workflow.md](/.claude/pgo-workflow.md)                                                                                                                 | LTO, PGO, SIMD, GPU strategies (Issue #96)                                      |
| Streaming sidecar (P0)       | [docs/development/ISSUE-107-SIDECAR-RELIABILITY.md](/docs/development/ISSUE-107-SIDECAR-RELIABILITY.md)                                                             | Watchdog, error recovery, systemd auto-restart                                  |
| Project context              | [docs/CONTEXT.md](/docs/CONTEXT.md)                                                                                                                                 | Why this project exists                                                         |
| API reference                | [docs/api/INDEX.md](/docs/api/INDEX.md)                                                                                                                             | Full Python API docs                                                            |
| Research                     | [docs/research/INDEX.md](/docs/research/INDEX.md)                                                                                                                   | ML labeling, regime patterns, TDA                                               |
| Oracle verification          | [docs/verification/](/docs/verification/)                                                                                                                           | Bit-exact cross-reference reports                                               |
| Source validation            | [.claude/skills/source-validation/SKILL.md](/.claude/skills/source-validation/SKILL.md)                                                                             | Vision→REST→WS junction proof, fromId pagination                                |
| Deploy secrets (local.env)   | `deploy/opendeviationbar.local.env`                                                                                                                                 | Survives git reset, holds tokens                                                |
| Heartbeat system             | `scripts/health_heartbeat.py`                                                                                                                                       | Ephemeral Telegram heartbeat via systemd                                        |
| Historical backfill          | `mise run kintsugi:catchup`                                                                                                                                         | Kintsugi catch-up for historical backfill                                       |
| Autonomous agent ops         | [.claude/skills/autonomous-agent-ops/SKILL.md](/.claude/skills/autonomous-agent-ops/SKILL.md)                                                                       | Multi-agent monitoring, pueue guardrails, watchdogs                             |
| Enable/add a symbol          | [.claude/skills/symbol-onboarding/SKILL.md](/.claude/skills/symbol-onboarding/SKILL.md)                                                                             | 9 call sites, sidecar.env gate, backfill, monitoring                            |
| Sidecar restart recovery     | [.claude/skills/sidecar-restart-zero-overlap-zero-gap-recovery-playbook/SKILL.md](/.claude/skills/sidecar-restart-zero-overlap-zero-gap-recovery-playbook/SKILL.md) | Stop→Delete→Backfill→Start, anti-patterns, L5-L9                                |

---

## Critical Principles

### 1. Leverage Rust

**ALWAYS use local Rust crates for heavy lifting.** Python handles I/O only.

1. **Check Rust first**: Before writing Python code, check if `opendeviationbar-core` already provides the capability
2. **Stream to Rust**: The `OpenDeviationBarProcessor` maintains state between `process_trades()` calls
3. **Checkpoint API**: Use `create_checkpoint()` and `from_checkpoint()` for cross-session continuity
4. **No reinventing**: Don't reimplement open deviation bar logic in Python

```python
# CORRECT: Stream to Rust (maintains state automatically)
for chunk in data_stream:
    bars = processor.process_trades(chunk.to_dicts())

# WRONG: Buffer in Python (OOM risk)
all_data = []
for chunk in data_stream:
    all_data.extend(chunk)
```

### 2. Kintsugi-First Gap Filling

**Kintsugi is the sole gap-filling mechanism.** All historical backfill flows through kintsugi with recency-first ordering (newest gaps first). Direct `populate_cache_resumable()` is guarded to max 366 days.

- **Daemon**: `mise run kintsugi:daemon` or `mise run kintsugi:catchup` (identical; `Conflicts=` prevents concurrent)
- **Full details**: [scripts/CLAUDE.md](/scripts/CLAUDE.md#kintsugi-sole-gap-filling-mechanism)

### 3. Symbol Registry Gate

**Every symbol must be registered before processing.** Unregistered symbols raise `SymbolNotRegisteredError`.

- **SSoT**: `symbols.toml` (repo root symlink → `python/opendeviationbar/data/symbols.toml`)
- **Adding new symbols**: Edit `symbols.toml`, run `maturin develop`, then process
- **Override** (dev only): `export OPENDEVIATIONBAR_SYMBOL_GATE=off`
- **Full details**: [python/opendeviationbar/CLAUDE.md](/python/opendeviationbar/CLAUDE.md#symbol-registry-issue-79)

### 4. Aion Continuity

**Aion is the only active mode.** Bars stream continuously with no resets. The processor resumes from the last committed trade ID in ClickHouse. Day-mode is fully deprecated (was used for forex, now disabled).

### 5. Writer Ownership Model (#280)

**Sidecar owns the live stream (resume TID forward). Kintsugi owns historical gaps.** Clean boundary eliminates concurrent-writer races. Asclepius REMOVED (#284) — unsafe for old gaps on a live engine.

- **Full details**: [scripts/CLAUDE.md](/scripts/CLAUDE.md#writer-ownership-model-280)
- **Restart recovery playbook**: [sidecar-restart-zero-overlap-zero-gap-recovery-playbook](/.claude/skills/sidecar-restart-zero-overlap-zero-gap-recovery-playbook/SKILL.md) — Stop→Delete→Backfill→Start, anti-patterns, defense layers L5-L9

### 6. Sidecar Startup Sequence Invariant (#294, #295)

**`_create_engine()` MUST follow this exact ordering:**

1. Start WS (buffered mode)
2. **Query per-threshold resume TIDs** from CH (`max(last_agg_trade_id)` per `(symbol, threshold)`)
3. **Load repair processor checkpoints** — if `~/.cache/opendeviationbar/repair_checkpoints/{SYM}_{THR}.json` exists, inject via `engine.set_checkpoint()`. With checkpoints: use `min()` across thresholds for REST start. Without: use `max()` (fewer junction gaps)
4. **DELETE bars after resume_tid** — skip-if-clean pre-check per partition (`mutations_sync=1`)
5. `fill_from_rest` — processors restored from checkpoints produce bars with matching TID boundaries (zero junction gaps)
6. `sync_flush` — INSERT new bars to CH

**CRITICAL**: Step 4 uses `ALTER TABLE DELETE IN PARTITION` (NOT lightweight `DELETE FROM`) with `mutations_sync=1`. Traditional mutations physically rewrite parts — no `_row_exists=0` mask persistence. Pre-fill timing is safe: the sidecar hasn't started streaming yet.

- **Dedup floor**: `committed_floors` initialized from `processor.last_completed_bar_tid()` (v1.4), NOT `last_agg_trade_id()` which includes forming bar tail
- **Checkpoint handoff** (#317): `repair_direct_parquet.py` saves processor checkpoints (forming bar state) at end of each symbol. Sidecar loads them → zero junction gaps between repair and sidecar bars

### 7. ClickHouse DELETE: Three Strategies (#295, v13.49.0)

Three DELETE strategies ranked fastest to slowest. **Lightweight `DELETE FROM` is permanently banned** — it creates `_row_exists=0` ghost rows that cause false Stathera anomalies in non-FINAL queries.

| Context              | Strategy              | Syntax                                                       | Speed                    | When to use                                 |
| -------------------- | --------------------- | ------------------------------------------------------------ | ------------------------ | ------------------------------------------- |
| **Bulk cleanup**     | `DROP PARTITION`      | `ALTER TABLE t DROP PARTITION (sym, thr)`                    | Instant (metadata-only)  | Purge entire symbol+threshold, test data    |
| **Sidecar pre-fill** | `DELETE IN PARTITION` | `ALTER TABLE t DELETE IN PARTITION (sym, thr) WHERE tid>...` | Fast (scans 1 partition) | TID-based clean-slate, targeted row removal |
| **Kintsugi replace** | `DELETE IN PARTITION` | Same, with `mutations_sync=1`                                | Fast (scans 1 partition) | Shard-recompute bar replacement             |
| **Manual recovery**  | `DELETE IN PARTITION` | Same, with `mutations_sync=1`                                | Fast (scans 1 partition) | Pattern 2 one-time operations               |
| **Fallback only**    | `ALTER TABLE DELETE`  | `ALTER TABLE t DELETE WHERE ...`                             | Slow (scans all parts)   | Only when partition unknown                 |

**Anti-pattern (NEVER USE):** `DELETE FROM t WHERE ...` — lightweight DELETE creates `_row_exists=0` masks that persist as ghost rows until CH background merge. Causes false Stathera gaps in heartbeat queries. Banned since v13.49.0 (root cause: #269 commit c9cb5c5 introduced it, phantom gaps reported for months).

- **Partition key**: `(symbol, threshold_decimal_bps)` — 64-72 partitions total. Time pruning via ORDER BY index.
- **All DELETEs require**: `mutations_sync=1` (synchronous, no async race)
- **Overlap self-heal DISABLED**: `_check_and_heal_overlaps()` removed — it amplified cascades

### 7. Spot-Only Data Source Policy

**All Binance data uses spot market exclusively. No futures.** REST (`api.binance.com/api/v3`), Vision (`data.binance.vision/data/spot/`), WebSocket (`stream.binance.com`). REST and Vision produce identical day boundaries (verified empirically). Use combined WS streams (2x9 hybrid), NOT per-symbol connections.

- **Full details**: [crates/opendeviationbar-providers/CLAUDE.md](/crates/opendeviationbar-providers/CLAUDE.md#spot-only-data-source-policy)

---

## Python API

```python
from opendeviationbar import get_open_deviation_bars, get_n_open_deviation_bars, process_trades_polars

# Date-bounded (backtesting)
df = get_open_deviation_bars("BTCUSDT", "2024-01-01", "2024-06-30")

# Count-bounded (ML training)
df = get_n_open_deviation_bars("BTCUSDT", n_bars=10000)

# Polars users (2-3x faster)
import polars as pl
trades = pl.scan_parquet("trades.parquet")
df = process_trades_polars(trades, threshold_decimal_bps=250)

# With microstructure features
df = get_open_deviation_bars("BTCUSDT", "2024-01-01", "2024-06-30", include_microstructure=True)

# Long ranges (>30 days) — must populate cache first
from opendeviationbar import populate_cache_resumable
populate_cache_resumable("BTCUSDT", "2019-01-01", "2025-12-31")
df = get_open_deviation_bars("BTCUSDT", "2019-01-01", "2025-12-31")
```

| API                           | Use Case                   | Details                                                                                                   |
| ----------------------------- | -------------------------- | --------------------------------------------------------------------------------------------------------- |
| `get_open_deviation_bars()`   | Date range, backtesting    | [docs/api/primary-api.md](/docs/api/primary-api.md)                                                       |
| `get_n_open_deviation_bars()` | Exact N bars, ML           | [docs/api/primary-api.md](/docs/api/primary-api.md)                                                       |
| `populate_cache_resumable()`  | Single segment (≤366 days) | [docs/api/cache-api.md](/docs/api/cache-api.md) — use `mise run kintsugi:catchup` for historical backfill |
| `process_trades_polars()`     | Polars DataFrames          | [docs/api/processing-api.md](/docs/api/processing-api.md)                                                 |
| `process_trades_chunked()`    | Large datasets >10M trades | [docs/api/processing-api.md](/docs/api/processing-api.md)                                                 |

**Full Python API details**: [python/opendeviationbar/CLAUDE.md](/python/opendeviationbar/CLAUDE.md)

---

## Architecture

```
opendeviationbar-py/
├── crates/                    10 Rust crates (5 on crates.io, 5 internal)
│   └── opendeviationbar-core/         Core algorithm, microstructure features
├── src/                       PyO3 bindings (Issue #94: domain modules)
│   ├── lib.rs                 Thin orchestrator (~200 lines)
│   ├── helpers.rs             Conversion functions
│   ├── core_bindings.rs       PyOpenDeviationBarProcessor, PyPositionVerification
│   ├── arrow_bindings.rs      Arrow export functions
│   ├── binance_bindings.rs    Binance data fetching
│   ├── exness_bindings.rs     Exness data fetching
│   ├── streaming_bindings.rs  LiveBarEngine, streaming classes
│   └── stream_manager_bindings.rs  StreamManager (Issue #161)
├── python/opendeviationbar/           Python API layer
│   ├── clickhouse/            ClickHouse cache (bigblack)
│   ├── sidecar.py             Streaming sidecar orchestrator (v12.20+)
│   ├── sidecar/arrow_flush.py     Arrow FFI drain + Polars zero-copy flush (v7.2)
│   ├── sidecar_api.py         Consumer API: ariadne, gap-fill, catchup (#213, #224)
│   ├── validation/            Microstructure validation
│   └── storage/               Tier 1 cache (Parquet)
├── scripts/                   Pueue jobs, validation, cache population, Exness seeder
├── .cargo/config.toml         Cross-compile friendly rustflags
└── pyproject.toml             Maturin config
```

**Key files**: `src/lib.rs` (PyO3 module registration), `python/opendeviationbar/__init__.py` (public API), `crates/opendeviationbar-core/src/processor.rs` (core algorithm), `python/opendeviationbar/sidecar.py` (streaming sidecar), `python/opendeviationbar/sidecar_api.py` (consumer API: ariadne + gap-fill + catchup)

**Full architecture**: [docs/ARCHITECTURE.md](/docs/ARCHITECTURE.md)

---

## Development Commands

```bash
# Setup (mise manages all tools including zig)
mise install

# Build & test
mise run build              # maturin develop
mise run test               # Rust tests (cargo nextest)
mise run test-py            # Python tests (pytest)

# Quality
mise run check-full         # fmt + lint + test + deny

# Release (see docs/development/RELEASE.md)
mise run release:full       # Full 4-phase workflow
mise run release:linux      # Zig cross-compile (~55 sec)
mise run release:pypi         # Upload to PyPI

# Benchmarks
mise run bench:run          # Full benchmarks
mise run bench:validate     # Verify 1M ticks < 100ms
```

---

## Build System

Zig cross-compilation from macOS (no remote SSH needed).

| Platform     | Strategy          | Time    | Command                        |
| ------------ | ----------------- | ------- | ------------------------------ |
| macOS ARM64  | Native maturin    | ~10 sec | `mise run release:macos-arm64` |
| Linux x86_64 | Zig cross-compile | ~55 sec | `mise run release:linux`       |

**Why rustls**: Eliminates OpenSSL dependency, enabling pure Rust cross-compilation.

**Full release workflow**: [docs/development/RELEASE.md](/docs/development/RELEASE.md)

---

## Key Subsystems (Progressive Disclosure)

### Microstructure Features

10 intra-bar + 16 inter-bar lookback features computed in Rust during bar construction. **Formulas, ranges, academic backing**: [crates/CLAUDE.md](/crates/CLAUDE.md#microstructure-features-v70). **Validation**: [python/opendeviationbar/CLAUDE.md](/python/opendeviationbar/CLAUDE.md#validation-framework-v70).

### ClickHouse Infrastructure

All data served from **bigblack** (remote GPU host, ~42M bars). Connection mode: `OPENDEVIATIONBAR_MODE=remote`. **Schema, population, dedup hardening**: [python/opendeviationbar/clickhouse/CLAUDE.md](/python/opendeviationbar/clickhouse/CLAUDE.md). **Plugin columns**: [python/opendeviationbar/plugins/CLAUDE.md](/python/opendeviationbar/plugins/CLAUDE.md). **Tier-1 Parquet ticks cache**: [python/opendeviationbar/storage/CLAUDE.md](/python/opendeviationbar/storage/CLAUDE.md).

### Monitoring & Gap Prevention

Defense-in-depth against silent data loss (born from Feb 2026 incident). 8 protection layers, Telegram alerts (@obdpy_bot), Monit watchdog (11 monitors), systemd heartbeat timer (5 min).

| Layer | Mechanism                                                            | Key File                       |
| ----- | -------------------------------------------------------------------- | ------------------------------ |
| L1    | Sidecar stream continuity (Aion resume from last committed TID)      | `sidecar.py`                   |
| L2    | Pre-write gate (filter+warn)                                         | `bulk_operations.py`           |
| L3    | Charon dead-letter cleanup                                           | `kintsugi.py`                  |
| L4    | Redis write lock per (symbol, threshold)                             | `write_lock.py`                |
| L5    | Pre-fill DELETE after resume TID (`ALTER TABLE DELETE IN PARTITION`) | `sidecar.py`                   |
| L6    | Heartbeat: unified per-symbol Stathera + today-gap LOUD alert        | `heartbeat/checks/stathera.py` |
| L7    | `last_completed_bar_tid` dedup floor (not forming bar tail)          | `live_engine.rs`               |
| L8    | Seeder Parquet cache maintenance                                     | `tick_cache_seeder.py`         |

**Full details**: [deploy/CLAUDE.md](/deploy/CLAUDE.md#monitoring--data-integrity). **Heartbeat & gap detection**: [scripts/CLAUDE.md](/scripts/CLAUDE.md#gap-detection--monitoring). **systemd services**: [scripts/systemd/CLAUDE.md](/scripts/systemd/CLAUDE.md).

### Streaming Sidecar

Layered self-healing architecture: Puck (inline gap fill), Niffler (immediate dead-letter replay), Charon (periodic dead-letter ferry), Argus (per-symbol liveness), Lethe (ring buffer overflow detection), Antaeus (resilient WS reconnection). **Full concept table + config**: [scripts/CLAUDE.md](/scripts/CLAUDE.md#streaming-reliability-concepts).

v7.2 adds Arrow FFI drain (`drain_bars_arrow()`) for zero-copy sync_flush, reducing peak RSS from ~2.6G to ~0.9G. **Memory efficiency**: [python/opendeviationbar/sidecar/CLAUDE.md](/python/opendeviationbar/sidecar/CLAUDE.md#v72-memory-efficiency).

### Common Errors & Breaking Changes

Troubleshooting table for frequent errors (`SymbolNotRegisteredError`, `High < Low`, OOM, fd exhaustion). Breaking changes: `gap_fill_trades()` response format (v12.62+), server-side catchup API (#224-#228). **Full details**: [python/opendeviationbar/CLAUDE.md](/python/opendeviationbar/CLAUDE.md#error-handling).

---

## CLAUDE.md Network (Hub-and-Spoke)

This file is the **hub**. Each spoke CLAUDE.md is loaded automatically when working in that directory.

| Directory                                 | CLAUDE.md                                                                            | SSoT For                                                                               |
| ----------------------------------------- | ------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------- |
| `/`                                       | This file                                                                            | Project overview, API, architecture, principles                                        |
| `/crates/`                                | [crates/CLAUDE.md](/crates/CLAUDE.md)                                                | Rust crates, microstructure formulas, algorithm details                                |
| `/crates/opendeviationbar-providers/`     | [opendeviationbar-providers/CLAUDE.md](/crates/opendeviationbar-providers/CLAUDE.md) | Binance Vision CSV format, data source policy, WS architecture                         |
| `/python/opendeviationbar/`               | [python/opendeviationbar/CLAUDE.md](/python/opendeviationbar/CLAUDE.md)              | Python API, caching, validation, symbol registry, memory guards                        |
| `/python/opendeviationbar/checkpoint/`    | [checkpoint/CLAUDE.md](/python/opendeviationbar/checkpoint/CLAUDE.md)                | Resumable cache population: engine, models, storage, cross-machine resume              |
| `/python/opendeviationbar/clickhouse/`    | [clickhouse/CLAUDE.md](/python/opendeviationbar/clickhouse/CLAUDE.md)                | ClickHouse schema (core only), dedup hardening, cache population                       |
| `/python/opendeviationbar/compat/`        | [compat/CLAUDE.md](/python/opendeviationbar/compat/CLAUDE.md)                        | Alpha-forge compatibility: panel format, feature manifest, availability                |
| `/python/opendeviationbar/config/`        | [config/CLAUDE.md](/python/opendeviationbar/config/CLAUDE.md)                        | Configuration via pydantic-settings, env var priority                                  |
| `/python/opendeviationbar/kintsugi/`      | [kintsugi/CLAUDE.md](/python/opendeviationbar/kintsugi/CLAUDE.md)                    | Gap reconciliation: discovery, repair, verification, daemon orchestration              |
| `/python/opendeviationbar/notify/`        | [notify/CLAUDE.md](/python/opendeviationbar/notify/CLAUDE.md)                        | Telegram notifications via @obdpy_bot, production alerts                               |
| `/python/opendeviationbar/orchestration/` | [orchestration/CLAUDE.md](/python/opendeviationbar/orchestration/CLAUDE.md)          | Rust→Python→ClickHouse pipeline, circuit breaker, count-bounded, tick fetch            |
| `/python/opendeviationbar/plugins/`       | [plugins/CLAUDE.md](/python/opendeviationbar/plugins/CLAUDE.md)                      | FeatureProvider protocol, plugin schema migration, entry-point discovery               |
| `/python/opendeviationbar/processors/`    | [processors/CLAUDE.md](/python/opendeviationbar/processors/CLAUDE.md)                | Trade processing pipeline: PyO3 wrapper, pandas/Polars/iterator entry points           |
| `/python/opendeviationbar/ratelimit/`     | [ratelimit/CLAUDE.md](/python/opendeviationbar/ratelimit/CLAUDE.md)                  | File-based cross-process rate limit coordinator, 6000 weight/min budget                |
| `/python/opendeviationbar/sidecar/`       | [sidecar/CLAUDE.md](/python/opendeviationbar/sidecar/CLAUDE.md)                      | Streaming sidecar: WS→Rust→ClickHouse, Arrow FFI v7.2, startup invariant, self-healing |
| `/python/opendeviationbar/storage/`       | [storage/CLAUDE.md](/python/opendeviationbar/storage/CLAUDE.md)                      | Tier-1 Parquet ticks cache: TickStorage, MEM-001/MEM-004, daily partitions             |
| `/python/opendeviationbar/validation/`    | [validation/CLAUDE.md](/python/opendeviationbar/validation/CLAUDE.md)                | 4-tier validation framework, Stathera continuity, gap classification                   |
| `/deploy/`                                | [deploy/CLAUDE.md](/deploy/CLAUDE.md)                                                | Env files SSoT, monit watchdog config, defense-in-depth, deploy workflow               |
| `/scripts/`                               | [scripts/CLAUDE.md](/scripts/CLAUDE.md)                                              | Kintsugi, writer ownership, parallelism, Parquet seeder, Stathera                      |
| `/scripts/heartbeat/`                     | [heartbeat/CLAUDE.md](/scripts/heartbeat/CLAUDE.md)                                  | Health monitoring: 5-min systemd timer, 4-tier severity, Telegram alerts               |
| `/scripts/heartbeat/checks/`              | [heartbeat/checks/CLAUDE.md](/scripts/heartbeat/checks/CLAUDE.md)                    | 14 health check modules, shared results dict interface, per-check documentation        |
| `/scripts/systemd/`                       | [scripts/systemd/CLAUDE.md](/scripts/systemd/CLAUDE.md)                              | 6 services, 3 timers, TZ=UTC, OOM protection, dependency chain, local.env              |
| `/src/`                                   | [src/CLAUDE.md](/src/CLAUDE.md)                                                      | PyO3 bindings: 8 domain modules, feature gates, Rust-to-Python bridge                  |

---

## Terminology (Essential)

| Term                   | Definition                                                                                                                                                                                                                   |
| ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **dbps**               | 1 dbps = 0.00001 = 0.001%. Example: 250 dbps = 0.25%. **All threshold values use dbps.**                                                                                                                                     |
| **Ouroboros**          | Bar reset mode framework. **Aion** (default, only active mode): continuous stream, no resets, bars span across days. **Week** (future): weekly reset boundary. Day-mode is fully deprecated.                                 |
| **Stathera**           | (σταθερά) Trade-ID alignment invariant: `bar[i].last_agg_trade_id + 1 == bar[i+1].first_agg_trade_id`. Replaces all time-based gap detection.                                                                                |
| **Kintsugi**           | Self-healing gap reconciliation. Discovers Stathera gaps via trade-ID audit and repairs using Ariadne pagination + Ouroboros.                                                                                                |
| **Shard**              | A Stathera gap — consecutive bars with `tid_delta > 0`. Identified by `anchor_last_tid` / `resume_first_tid`.                                                                                                                |
| **Puck**               | Inline Rust gap fill on every WS trade. Detects trade-ID discontinuity, fills in sub-3s via parallel REST fetch.                                                                                                             |
| **Circuit Breaker**    | `fatal_write_breaker`: CLOSED → (3 failures) → OPEN → (60s) → HALF_OPEN → (success) → CLOSED. Prevents cascade during ClickHouse outages.                                                                                    |
| **committed_floors**   | Per-threshold dedup floor in `live_engine.rs`. Bars with `last_agg_trade_id <= floor` are skipped. Initialized from `last_completed_bar_tid()` (v1.4), NOT `last_agg_trade_id()`.                                            |
| **Clean-slate DELETE** | Sidecar startup: `ALTER TABLE DELETE IN PARTITION` bars after resume TID BEFORE `fill_from_rest`. Per-threshold TID-based DELETE ensures fresh recompute from resume point.                                                  |
| **`mutations_sync=1`** | ClickHouse setting that makes DELETE synchronous. REQUIRED on all DELETE operations (both lightweight and traditional). Used by sidecar (traditional pre-fill) and kintsugi (lightweight) and manual recovery (traditional). |

**Full terminology** (15 named concepts including Ariadne, Antaeus, Lethe, Argus, Charon, Niffler, StreamManager, AdaptiveRateLimiter, Shadow Validation): [scripts/CLAUDE.md](/scripts/CLAUDE.md#streaming-reliability-concepts)

<!-- gitnexus:start -->

# GitNexus MCP

This project is indexed by GitNexus as **opendeviationbar-py** (8147 symbols, 21188 relationships, 300 execution flows).

## Always Start Here

1. **Read `gitnexus://repo/{name}/context`** — codebase overview + check index freshness
2. **Match your task to a skill below** and **read that skill file**
3. **Follow the skill's workflow and checklist**

> If step 1 warns the index is stale, run `npx gitnexus analyze` in the terminal first.

## Skills

| Task                                         | Read this skill file                                        |
| -------------------------------------------- | ----------------------------------------------------------- |
| Understand architecture / "How does X work?" | `.claude/skills/gitnexus/gitnexus-exploring/SKILL.md`       |
| Blast radius / "What breaks if I change X?"  | `.claude/skills/gitnexus/gitnexus-impact-analysis/SKILL.md` |
| Trace bugs / "Why is X failing?"             | `.claude/skills/gitnexus/gitnexus-debugging/SKILL.md`       |
| Rename / extract / split / refactor          | `.claude/skills/gitnexus/gitnexus-refactoring/SKILL.md`     |
| Tools, resources, schema reference           | `.claude/skills/gitnexus/gitnexus-guide/SKILL.md`           |
| Index, status, clean, wiki CLI commands      | `.claude/skills/gitnexus/gitnexus-cli/SKILL.md`             |

<!-- gitnexus:end -->
