---
phase: 05-microsecond-clickhouse-rebuild
plan: 01
type: execute
wave: 1
depends_on: []
files_modified:
  - src/helpers.rs
  - python/opendeviationbar/constants.py
  - python/opendeviationbar/validation/schemas.py
  - python/opendeviationbar/validation/day_mode/guards.py
  - python/opendeviationbar/orchestration/helpers.py
  - python/opendeviationbar/backfill.py
  - python/opendeviationbar/rectify.py
  - python/opendeviationbar/processors/core.py
  - python/opendeviationbar/exness.py
  - python/opendeviationbar/config/backfill.py
  - python/opendeviationbar/checkpoint/models.py
  - python/opendeviationbar/checkpoint/storage.py
  - tests/test_day_mode_invariant.py
  - tests/test_data_contracts.py
  - tests/test_streaming.py
  - tests/test_rust_bindings.py
  - tests/test_exchange_sessions.py
  - tests/test_multi_threshold.py
  - tests/test_overlap_strategy.py
  - tests/test_sync_flush_atomic_replace.py
  - tests/test_ariadne_telemetry.py
  - tests/test_oracle_regression.py
  - tests/test_heal_scoping.py
  - tests/test_rectification.py
  - tests/test_checkpoint_edge_cases.py
  - tests/test_oversized_bar_guard.py
autonomous: true
requirements: [USEC-05]

must_haves:
  truths:
    - "Rust opendeviationbar_to_dict() outputs open_time_us/close_time_us keys with raw microsecond values (no /1000 division)"
    - "DAY_US = 86_400_000_000 replaces DAY_MS in guards.py and all importers"
    - "BarOutputSchema validates open_time_us/close_time_us with microsecond bounds (1e15..2.5e15)"
    - "TIMESTAMP_COLUMNS tuple contains close_time_us and open_time_us"
    - "All 14 test files updated to use _us column names and microsecond values"
  artifacts:
    - path: "src/helpers.rs"
      provides: "us-native dict output from Rust"
      contains: "open_time_us"
    - path: "python/opendeviationbar/validation/day_mode/guards.py"
      provides: "DAY_US constant"
      contains: "DAY_US"
    - path: "python/opendeviationbar/validation/schemas.py"
      provides: "us-native BarOutputSchema"
      contains: "open_time_us"
    - path: "python/opendeviationbar/constants.py"
      provides: "TIMESTAMP_COLUMNS with _us names"
      contains: "close_time_us"
  key_links:
    - from: "src/helpers.rs"
      to: "python/opendeviationbar/validation/schemas.py"
      via: "dict key names must match schema field names"
      pattern: "open_time_us"
    - from: "python/opendeviationbar/validation/day_mode/guards.py"
      to: "python/opendeviationbar/backfill.py"
      via: "DAY_US import"
      pattern: "from.*guards import DAY_US"
---

<objective>
Migrate the foundational timestamp contracts from milliseconds to microseconds: Rust dict output, Python constants, Pandera schema, day-mode guards, and all 14 test files.

Purpose: Establishes the new \_us column name convention and microsecond arithmetic constants that all downstream code (ClickHouse layer, sidecar, kintsugi, heartbeat) will depend on in subsequent plans.
Output: Rust extension outputs us-native dicts, Python contracts enforce us bounds, all tests pass with us conventions.
</objective>

<execution_context>
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/05-microsecond-clickhouse-rebuild/05-RESEARCH.md

<interfaces>
<!-- Key types and contracts the executor needs -->

From src/helpers.rs (lines 102-103, current ms output):

```rust
dict.set_item(ikey(py, "open_time_ms"), bar.open_time / 1000)?;
dict.set_item(ikey(py, "close_time_ms"), bar.close_time / 1000)?;
```

From python/opendeviationbar/validation/schemas.py (current ms bounds):

```python
_TS_MS_MIN = 1_000_000_000_000
_TS_MS_MAX = 2_500_000_000_000
class BarOutputSchema(pa.DataFrameModel):
    open_time_ms: int = pa.Field(ge=_TS_MS_MIN, le=_TS_MS_MAX)
    close_time_ms: int = pa.Field(ge=_TS_MS_MIN, le=_TS_MS_MAX)
```

From python/opendeviationbar/validation/day_mode/guards.py:

```python
DAY_MS = 86_400_000  # milliseconds per UTC day
```

From python/opendeviationbar/constants.py (line 191):

```python
TIMESTAMP_COLUMNS: tuple[str, ...] = ("close_time_ms", "open_time_ms",)
```

</interfaces>
</context>

<tasks>

<task type="auto">
  <name>Task 1: Migrate Rust dict output and Python contracts to microseconds</name>
  <files>
    src/helpers.rs,
    python/opendeviationbar/constants.py,
    python/opendeviationbar/validation/schemas.py,
    python/opendeviationbar/validation/day_mode/guards.py,
    python/opendeviationbar/orchestration/helpers.py,
    python/opendeviationbar/backfill.py,
    python/opendeviationbar/rectify.py,
    python/opendeviationbar/processors/core.py,
    python/opendeviationbar/exness.py,
    python/opendeviationbar/config/backfill.py,
    python/opendeviationbar/checkpoint/models.py,
    python/opendeviationbar/checkpoint/storage.py
  </files>
  <read_first>
    src/helpers.rs,
    python/opendeviationbar/constants.py,
    python/opendeviationbar/validation/schemas.py,
    python/opendeviationbar/validation/day_mode/guards.py,
    python/opendeviationbar/orchestration/helpers.py,
    python/opendeviationbar/backfill.py,
    python/opendeviationbar/rectify.py,
    python/opendeviationbar/processors/core.py,
    python/opendeviationbar/exness.py,
    python/opendeviationbar/config/backfill.py,
    python/opendeviationbar/checkpoint/models.py,
    python/opendeviationbar/checkpoint/storage.py
  </read_first>
  <action>
    **Rust (src/helpers.rs):**
    - Line 102: Change `dict.set_item(ikey(py, "open_time_ms"), bar.open_time / 1000)?;` to `dict.set_item(ikey(py, "open_time_us"), bar.open_time)?;` — remove the /1000 division, rename key to _us
    - Line 103: Change `dict.set_item(ikey(py, "close_time_ms"), bar.close_time / 1000)?;` to `dict.set_item(ikey(py, "close_time_us"), bar.close_time)?;` — remove the /1000 division, rename key to _us
    - Also check for `first_trade_time_ms` and `last_trade_time_ms` dict keys in the same function and rename those to `_us` with no division if they exist

    **Python constants (constants.py):**
    - Change `TIMESTAMP_COLUMNS` tuple from `("close_time_ms", "open_time_ms")` to `("close_time_us", "open_time_us")`
    - Update the surrounding comment from "close_time_ms replaces..." to "close_time_us replaces..."
    - Update the IMPORTANT sync comment to reference _us columns

    **Pandera schema (schemas.py):**
    - Rename `_TS_MS_MIN`/`_TS_MS_MAX` to `_TS_US_MIN`/`_TS_US_MAX`
    - Set `_TS_US_MIN = 1_000_000_000_000_000` (1e15, ~2001-09 in us)
    - Set `_TS_US_MAX = 2_500_000_000_000_000` (2.5e15, ~2049-03 in us)
    - Rename fields: `open_time_ms` -> `open_time_us`, `close_time_ms` -> `close_time_us`
    - Update all comments and docstring to say "microseconds" not "milliseconds"

    **Day-mode guards (guards.py):**
    - Rename `DAY_MS = 86_400_000` to `DAY_US = 86_400_000_000` (microseconds per UTC day)
    - Update `split_trades_by_day()`: change `day = ts_ms // DAY_MS` to `day = ts_us // DAY_US` and rename local var from `ts_ms` to `ts_us`
    - Update dict key lookups: `t.get("timestamp")` already returns us values from Rust, so arithmetic is correct. The `// DAY_US` handles the us division.
    - Also add a backward-compat alias: `DAY_MS = DAY_US // 1000` with a deprecation comment for any code that imports DAY_MS during transition — actually NO, do NOT add this. Clean break. grep for all DAY_MS imports and update them.

    **orchestration/helpers.py:**
    - Find the `open_time_ms` / `close_time_ms` alias expressions that divide by 1000 (lines ~334/338 per research: `(pl.col("open_time") // 1000).alias("open_time_ms")`)
    - Change to `pl.col("open_time").alias("open_time_us")` — no division, rename alias
    - Same for close_time

    **backfill.py:**
    - Change `from ...guards import DAY_MS` to `from ...guards import DAY_US`
    - Update all `DAY_MS` references to `DAY_US`
    - Update any `86_400_000` literal to `86_400_000_000`

    **rectify.py:**
    - Change `open_time_ms`/`close_time_ms` column references to `open_time_us`/`close_time_us`

    **processors/core.py:**
    - Change `pd.to_datetime(..., unit="ms")` to `pd.to_datetime(..., unit="us")` for any timestamp conversion

    **exness.py:**
    - Change `pd.to_datetime(..., unit="ms")` to `pd.to_datetime(..., unit="us")` for any timestamp conversion

    **config/backfill.py:**
    - Update comment `86400000 = 1 day` to `86400000000 = 1 day (microseconds)`

    **checkpoint/models.py + checkpoint/storage.py:**
    - Rename field `last_trade_timestamp_ms` to `last_trade_timestamp_us`
    - Update all references

    After all Python changes, run `maturin develop` to rebuild the Rust extension.

  </action>
  <verify>
    <automated>cd /Users/terryli/eon/opendeviationbar-py && maturin develop 2>&1 | tail -3</automated>
  </verify>
  <acceptance_criteria>
    - grep -r "open_time_ms" src/helpers.rs returns NO matches
    - grep "open_time_us" src/helpers.rs returns matches at the dict.set_item lines
    - grep "DAY_US" python/opendeviationbar/validation/day_mode/guards.py returns the constant definition
    - grep "DAY_MS" python/opendeviationbar/validation/day_mode/guards.py returns NO matches
    - grep "open_time_us" python/opendeviationbar/validation/schemas.py returns the field definition
    - grep "_TS_US_MIN" python/opendeviationbar/validation/schemas.py returns the bound
    - grep "close_time_us" python/opendeviationbar/constants.py returns the TIMESTAMP_COLUMNS entry
    - grep "DAY_MS" python/opendeviationbar/backfill.py returns NO matches
    - grep "last_trade_timestamp_us" python/opendeviationbar/checkpoint/models.py returns matches
  </acceptance_criteria>
  <done>Rust outputs us-native dict keys, Python contracts (constants, schemas, guards) all use _us naming and microsecond values, checkpoint models use _us field name, maturin build succeeds</done>
</task>

<task type="auto">
  <name>Task 2: Update all 14 test files from _ms to _us conventions</name>
  <files>
    tests/test_day_mode_invariant.py,
    tests/test_data_contracts.py,
    tests/test_streaming.py,
    tests/test_rust_bindings.py,
    tests/test_exchange_sessions.py,
    tests/test_multi_threshold.py,
    tests/test_overlap_strategy.py,
    tests/test_sync_flush_atomic_replace.py,
    tests/test_ariadne_telemetry.py,
    tests/test_oracle_regression.py,
    tests/test_heal_scoping.py,
    tests/test_rectification.py,
    tests/test_checkpoint_edge_cases.py,
    tests/test_oversized_bar_guard.py
  </files>
  <read_first>
    tests/test_day_mode_invariant.py,
    tests/test_data_contracts.py,
    tests/test_streaming.py,
    tests/test_rust_bindings.py,
    tests/test_exchange_sessions.py,
    tests/test_multi_threshold.py,
    tests/test_overlap_strategy.py,
    tests/test_sync_flush_atomic_replace.py,
    tests/test_ariadne_telemetry.py,
    tests/test_oracle_regression.py,
    tests/test_heal_scoping.py,
    tests/test_rectification.py,
    tests/test_checkpoint_edge_cases.py,
    tests/test_oversized_bar_guard.py
  </read_first>
  <action>
    For each of the 14 test files, apply these systematic changes:

    **Column name changes:**
    - `"open_time_ms"` -> `"open_time_us"` (string literals in dicts, DataFrames, assertions)
    - `"close_time_ms"` -> `"close_time_us"` (string literals)
    - `"first_trade_time_ms"` -> `"first_trade_time_us"` if present
    - `"last_trade_time_ms"` -> `"last_trade_time_us"` if present

    **Timestamp value changes:**
    - Any 13-digit millisecond timestamp literal (e.g., `1704067200000`) must be multiplied by 1000 to become a 16-digit microsecond value (e.g., `1704067200000000`)
    - `86_400_000` -> `86_400_000_000` (DAY_MS -> DAY_US)
    - `3_600_000` -> `3_600_000_000` (HOUR_MS -> HOUR_US)

    **Import changes:**
    - `from opendeviationbar.validation.day_mode.guards import DAY_MS` -> `import DAY_US`
    - Any local `DAY_MS` definitions in tests -> `DAY_US` with us value

    **pd.to_datetime changes:**
    - `pd.to_datetime(..., unit="ms")` -> `pd.to_datetime(..., unit="us")`

    **Schema reference changes:**
    - `_TS_MS_MIN`/`_TS_MS_MAX` -> `_TS_US_MIN`/`_TS_US_MAX` if imported in tests

    **CRITICAL**: Some tests may construct bar dicts directly with timestamp values. These must use 16-digit us values. Example: `{"open_time_us": 1704067200000000, "close_time_us": 1704067260000000}`.

    After updating all test files, run the test suite to verify.

  </action>
  <verify>
    <automated>cd /Users/terryli/eon/opendeviationbar-py && uv run --python 3.13 pytest tests/test_day_mode_invariant.py tests/test_data_contracts.py tests/test_rust_bindings.py tests/test_rectification.py tests/test_checkpoint_edge_cases.py -x --timeout=120 2>&1 | tail -20</automated>
  </verify>
  <acceptance_criteria>
    - grep -rn "open_time_ms" tests/ returns NO matches (across all 14 test files)
    - grep -rn "close_time_ms" tests/ returns NO matches
    - grep -rn "DAY_MS" tests/ returns NO matches
    - grep -rn "open_time_us" tests/ returns matches in the updated test files
    - pytest on the 5 core test files passes (test_day_mode_invariant, test_data_contracts, test_rust_bindings, test_rectification, test_checkpoint_edge_cases)
  </acceptance_criteria>
  <done>All 14 test files use _us column names and microsecond values, core tests pass with the new contracts</done>
</task>

</tasks>

<verification>
- `maturin develop` succeeds (Rust compiles with us changes)
- `grep -rn "open_time_ms\|close_time_ms\|DAY_MS" src/helpers.rs python/opendeviationbar/constants.py python/opendeviationbar/validation/ tests/` returns zero matches
- Core tests pass: `pytest tests/test_day_mode_invariant.py tests/test_data_contracts.py tests/test_rust_bindings.py -x`
</verification>

<success_criteria>

- Rust opendeviationbar_to_dict() outputs open_time_us/close_time_us with raw microsecond values
- DAY_US = 86_400_000_000 is the sole day constant (no DAY_MS in active code)
- BarOutputSchema validates microsecond bounds (1e15..2.5e15)
- TIMESTAMP_COLUMNS = ("close_time_us", "open_time_us")
- All 14 test files pass with \_us conventions
  </success_criteria>

<output>
After completion, create `.planning/phases/05-microsecond-clickhouse-rebuild/05-01-SUMMARY.md`
</output>
