---
phase: 34-core-clickhouse-writer
plan: 01
subsystem: streaming/clickhouse_writer
tags: [rust, clickhouse, types, dedup, streaming]
dependency_graph:
  requires: []
  provides:
    [
      ClickHouseBarRow,
      compute_dedup_token,
      ClickHouseWriterConfig,
      ClickHouseSummary,
    ]
  affects: [crates/opendeviationbar-streaming]
tech_stack:
  added: [clickhouse-rs-0.14, md5-0.8]
  patterns: [feature-gate, serde-custom-deserializer, env-var-config]
key_files:
  created:
    - crates/opendeviationbar-streaming/src/clickhouse_writer/mod.rs
    - crates/opendeviationbar-streaming/src/clickhouse_writer/row.rs
    - crates/opendeviationbar-streaming/src/clickhouse_writer/dedup.rs
    - crates/opendeviationbar-streaming/src/clickhouse_writer/config.rs
    - crates/opendeviationbar-streaming/src/clickhouse_writer/summary.rs
  modified:
    - Cargo.toml
    - crates/opendeviationbar-streaming/Cargo.toml
    - crates/opendeviationbar-streaming/src/lib.rs
decisions:
  - clickhouse-sink feature implies binance-integration (CompletedBar requires live_engine types)
  - ClickHouseBarRow omits is_liquidation_cascade and computed_at (CH DEFAULT expressions)
  - lookback_vwap_raw converted from Option<i64> to Option<f64> via SCALE division
  - ouroboros_mode hardcoded to "aion" (only active mode since v8.0)
metrics:
  duration: 854s
  completed: "2026-03-29T08:18:16Z"
  tasks_completed: 2
  tasks_total: 2
  tests_added: 20
  files_created: 5
  files_modified: 3
---

# Phase 34 Plan 01: Foundational Types Summary

ClickHouseBarRow struct mapping 60+ CompletedBar fields to CH columns, MD5 dedup token with Python hashlib parity, writer config with env var overrides, and X-ClickHouse-Summary JSON parser with string-encoded integer handling.

## Tasks Completed

### Task 1: Cargo deps + feature gate + module scaffold + ClickHouseBarRow + dedup token

- Added `clickhouse` (0.14, inserter+lz4) and `md5` (0.8) to workspace dependencies
- Created `clickhouse-sink` feature gate in opendeviationbar-streaming (implies `binance-integration`)
- `ClickHouseBarRow` struct with all schema columns EXCEPT `is_liquidation_cascade` and `computed_at`
- Critical field mappings: prices via `FixedPoint::to_f64()`, volumes via `i128 / SCALE`, `aggregation_density` from `aggregation_density_f64`
- `compute_dedup_token()` produces MD5 hex identical to Python `hashlib.md5()` (verified with fixed test vector)
- `compute_batch_dedup_token()` uses min(open_time_us), max(close_time_us) across batch
- `CORE_COLUMNS` constant lists all 78 column names for plugin discovery
- 14 unit tests: prices, volumes, lookback_vwap, aggregation_density, hardcoded fields, timestamps, trade IDs, dedup parity

**Commit:** 283f081

### Task 2: Config struct + X-ClickHouse-Summary parser

- `ClickHouseWriterConfig` with defaults: N=500 rows, T=100ms, bytes=5MB, channel=10K, retries=5
- `from_env()` reads `OPENDEVIATIONBAR_CH_HOSTS`, `OPENDEVIATIONBAR_CH_WRITER_MAX_ROWS/FLUSH_MS/MAX_BYTES`
- `ClickHouseSummary` with custom `deserialize_string_u64` for ClickHouse's string-encoded JSON values
- `parse_summary()` and `verify_written_rows()` with tracing::warn on mismatch
- 6 unit tests: defaults, env fallback, valid/invalid JSON parsing, match/mismatch verification

**Commit:** eaac118

## Deviations from Plan

### Auto-fixed Issues

**1. [Rule 3 - Blocking] clickhouse-sink feature requires binance-integration**

- **Found during:** Task 1
- **Issue:** `ClickHouseBarRow::from_completed_bar()` uses `crate::live_engine::CompletedBar` which is behind `binance-integration` feature gate
- **Fix:** Made `clickhouse-sink` feature imply `binance-integration` in Cargo.toml
- **Files modified:** `crates/opendeviationbar-streaming/Cargo.toml`

**2. [Rule 1 - Bug] Clippy collapsible_if warnings**

- **Found during:** Task 2
- **Issue:** Nested `if let` blocks in `from_env()` triggered clippy `collapsible_if` lint (streaming crate doesn't inherit workspace lint config)
- **Fix:** Used Rust 2024 `let-chains` syntax (`if let Ok(val) = ... && let Ok(n) = ...`)
- **Files modified:** `crates/opendeviationbar-streaming/src/clickhouse_writer/config.rs`

## Verification

- All 135 streaming crate tests pass (including 20 new clickhouse_writer tests)
- Workspace compiles without `clickhouse-sink` feature (feature gate works)
- Clippy passes with `-D warnings`
- Dedup token byte-identical to Python `hashlib.md5('BTCUSDT_250_1700000000000000_1700000100000000_aion')` = `1fd03e7c7957ee24987e9c710f82a7ca`

## Known Stubs

None - all types are fully implemented with no placeholder data.

## Self-Check: PASSED

- All 5 created files exist
- Both commits (283f081, eaac118) verified in git log
- All 13 acceptance criteria pass (is_liquidation_cascade/computed_at appear only in doc comments, not as struct fields)
