---
phase: 01-modularize-long-files-live-engine-rs-checkpoint-py-backfill-py
plan: 02
type: execute
wave: 1
depends_on: []
files_modified:
  - python/opendeviationbar/checkpoint.py
  - python/opendeviationbar/checkpoint/__init__.py
  - python/opendeviationbar/checkpoint/models.py
  - python/opendeviationbar/checkpoint/storage.py
  - python/opendeviationbar/checkpoint/engine.py
  - python/opendeviationbar/checkpoint.pyi
autonomous: true
requirements: []

must_haves:
  truths:
    - "All existing Python tests pass without modification"
    - "All imports from opendeviationbar.checkpoint work unchanged"
    - "populate_cache_resumable and populate_cache_multi_threshold callable from opendeviationbar package"
  artifacts:
    - path: "python/opendeviationbar/checkpoint/__init__.py"
      provides: "Public orchestrators + all re-exports for backward compatibility"
      contains: "populate_cache_resumable"
    - path: "python/opendeviationbar/checkpoint/models.py"
      provides: "PopulationContext and PopulationCheckpoint dataclasses"
      contains: "class PopulationContext"
    - path: "python/opendeviationbar/checkpoint/storage.py"
      provides: "Checkpoint path resolution, CH load/save, provenance"
      contains: "def _get_checkpoint_path"
    - path: "python/opendeviationbar/checkpoint/engine.py"
      provides: "_process_day, _process_one_threshold_day, _process_day_multi_threshold, _write_bars"
      contains: "def _process_day"
  key_links:
    - from: "python/opendeviationbar/checkpoint/__init__.py"
      to: "models.py, storage.py, engine.py"
      via: "from .models import, from .storage import, from .engine import"
      pattern: "from \\.models import"
    - from: "python/opendeviationbar/__init__.py"
      to: "checkpoint package"
      via: "from .checkpoint import"
      pattern: "from \\.checkpoint import populate_cache"
---

<objective>
Convert checkpoint.py (2,188 lines) from a single file into a checkpoint/ package with models.py, storage.py, engine.py, and __init__.py submodules.

Purpose: Improve maintainability by splitting dataclass models, storage I/O, and processing engine from the large orchestrator file. Follows the proven sidecar/ package conversion pattern.
Output: checkpoint/ package directory with 4 files, all public imports preserved, all tests passing.
</objective>

<execution_context>
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/01-modularize-long-files-live-engine-rs-checkpoint-py-backfill-py/01-CONTEXT.md

@python/opendeviationbar/checkpoint.py
@python/opendeviationbar/checkpoint.pyi
@python/opendeviationbar/**init**.py (lines 130-141 only)

<interfaces>
<!-- Public imports from checkpoint that MUST remain working -->
From python/opendeviationbar/__init__.py (line 140):
```python
from .checkpoint import populate_cache_multi_threshold, populate_cache_resumable
```

From python/opendeviationbar/sidecar/checkpoint.py (line 59):

```python
from opendeviationbar.checkpoint import _checkpoint_provenance
```

From python/opendeviationbar/kintsugi/daemon.py -- no direct checkpoint imports

<!-- checkpoint.pyi stub exports -->

```python
def populate_cache_resumable(...) -> int: ...
def populate_cache_multi_threshold(...) -> dict[int, int]: ...
```

</interfaces>
</context>

<tasks>

<task type="auto">
  <name>Task 1: Convert checkpoint.py to checkpoint/ package with submodules</name>
  <files>
    python/opendeviationbar/checkpoint.py
    python/opendeviationbar/checkpoint/__init__.py
    python/opendeviationbar/checkpoint/models.py
    python/opendeviationbar/checkpoint/storage.py
    python/opendeviationbar/checkpoint/engine.py
    python/opendeviationbar/checkpoint.pyi
  </files>
  <read_first>
    python/opendeviationbar/checkpoint.py
    python/opendeviationbar/checkpoint.pyi
    python/opendeviationbar/__init__.py (lines 130-141)
    python/opendeviationbar/sidecar/checkpoint.py (lines 55-65 -- import of _checkpoint_provenance)
    python/opendeviationbar/sidecar/__init__.py (first 30 lines -- package pattern)
  </read_first>
  <action>
    Convert `checkpoint.py` monolith into `checkpoint/` package. Follow the sidecar/ package pattern (directory with `__init__.py` re-exporting all public symbols).

    **Step 0: Rename** -- Delete `checkpoint.py` and create `checkpoint/` directory. Also handle `checkpoint.pyi` (it stays as `checkpoint.pyi` at the same level -- Python resolves package stubs from `__init__.pyi` inside the package, OR from a sibling `.pyi` file. Since the sidecar/ package uses `sidecar/__init__.pyi`, follow that same pattern: move `checkpoint.pyi` to `checkpoint/__init__.pyi`).

    **Step 1: Create models.py** (~220 lines) containing:
    - All imports needed by the dataclasses (json, logging, os, socket, tempfile, dataclasses, datetime, pathlib, typing, platformdirs)
    - `_CHECKPOINT_DIR` constant (line 51)
    - `def _get_population_config()` (lines 54-58)
    - `class PopulationContext` dataclass (lines 62-86)
    - `class PopulationCheckpoint` dataclass (lines 89-278) including all its methods: `from_dict()`, `to_dict()`, `save_local()`, `load_local()`
    - Logger: `logger = logging.getLogger(__name__)`

    **Step 2: Create storage.py** (~160 lines) containing:
    - `def _get_checkpoint_path(...)` (lines 280-320)
    - `def _load_checkpoint_from_clickhouse(...)` (lines 323-391)
    - `def _save_checkpoint_to_clickhouse(...)` (lines 392-428)
    - `def _checkpoint_provenance(...)` (lines 429-442)
    - Imports: `from .models import PopulationCheckpoint, _CHECKPOINT_DIR` plus logging, json, socket, datetime, pathlib, typing
    - Logger: `logger = logging.getLogger(__name__)`

    **Step 3: Create engine.py** (~500 lines) containing:
    - `def _is_ouroboros_boundary(...)` (lines 444-454)
    - `def _date_range(...)` (lines 456-479)
    - `def _write_bars(...)` (lines 482-528)
    - `def _process_day(...)` (lines 531-679)
    - `def _process_one_threshold_day(...)` (lines 681-792)
    - `def _process_day_multi_threshold(...)` (lines 794-941)
    - Imports: `from .models import PopulationContext, PopulationCheckpoint` plus gc, logging, os, datetime, typing, and lazy imports inside functions (same pattern as original)
    - Logger: `logger = logging.getLogger(__name__)`

    **Step 4: Create __init__.py** (~1200 lines) containing:
    - Module docstring (preserve from original lines 1-24)
    - `from __future__ import annotations`
    - Import and re-export from submodules:
      ```python
      from .models import PopulationCheckpoint, PopulationContext, _CHECKPOINT_DIR, _get_population_config
      from .storage import _get_checkpoint_path, _load_checkpoint_from_clickhouse, _save_checkpoint_to_clickhouse, _checkpoint_provenance
      from .engine import _is_ouroboros_boundary, _date_range, _write_bars, _process_day, _process_one_threshold_day, _process_day_multi_threshold
      ```
    - The remaining orchestrator functions that stay in __init__.py:
      - `def populate_cache_multi_threshold(...)` (lines 944-1396)
      - `def populate_cache_resumable(...)` (lines 1398-2103)
      - `def list_checkpoints(...)` (lines 2105-2133)
      - `def clear_checkpoint(...)` (lines 2135-2188)
    - These orchestrators import from submodules instead of having the functions inline

    **Step 5: Move checkpoint.pyi** to `checkpoint/__init__.pyi` (following sidecar/ pattern). Content stays the same -- it documents the public API of the package.

    **CRITICAL**:
    - All `from opendeviationbar.checkpoint import X` must still work
    - The `_checkpoint_provenance` private function must be importable from `opendeviationbar.checkpoint` (used by sidecar/checkpoint.py)
    - No behavioral changes -- pure code movement
    - Preserve all `# FILE-SIZE-OK` markers where appropriate

  </action>
  <verify>
    <automated>cd /Users/terryli/eon/opendeviationbar-py && python -c "from opendeviationbar.checkpoint import populate_cache_resumable, populate_cache_multi_threshold, PopulationCheckpoint, PopulationContext, _checkpoint_provenance; print('All imports OK')"</automated>
  </verify>
  <acceptance_criteria>
    - File `python/opendeviationbar/checkpoint.py` no longer exists (replaced by directory)
    - File `python/opendeviationbar/checkpoint/__init__.py` exists and contains `def populate_cache_resumable`
    - File `python/opendeviationbar/checkpoint/__init__.py` contains `from .models import`
    - File `python/opendeviationbar/checkpoint/models.py` exists and contains `class PopulationContext`
    - File `python/opendeviationbar/checkpoint/models.py` contains `class PopulationCheckpoint`
    - File `python/opendeviationbar/checkpoint/storage.py` exists and contains `def _get_checkpoint_path`
    - File `python/opendeviationbar/checkpoint/storage.py` contains `def _checkpoint_provenance`
    - File `python/opendeviationbar/checkpoint/engine.py` exists and contains `def _process_day`
    - File `python/opendeviationbar/checkpoint/engine.py` contains `def _write_bars`
    - File `python/opendeviationbar/checkpoint/__init__.pyi` exists and contains `def populate_cache_resumable`
    - `python -c "from opendeviationbar.checkpoint import populate_cache_resumable, PopulationCheckpoint, _checkpoint_provenance"` exits 0
    - `python -c "from opendeviationbar import populate_cache_resumable, populate_cache_multi_threshold"` exits 0
  </acceptance_criteria>
  <done>checkpoint.py split into 4 submodule files under checkpoint/ package, all imports preserved, all re-exports working</done>
</task>

<task type="auto">
  <name>Task 2: Run Python test suite to confirm zero regressions</name>
  <files>
    python/opendeviationbar/checkpoint/__init__.py
  </files>
  <read_first>
    python/opendeviationbar/checkpoint/__init__.py (first 30 lines -- verify re-exports)
  </read_first>
  <action>
    Run the Python test suite to confirm no regressions from the checkpoint.py package conversion.

    Run:
    1. `uv run --python 3.13 pytest tests/ -x -q --timeout=60` -- full Python test suite with fail-fast
    2. Specifically verify checkpoint-related tests pass: `uv run --python 3.13 pytest tests/ -x -q -k "checkpoint or populate" --timeout=60`

    If any test fails due to import issues:
    - Check if the test imports a private function that wasn't re-exported in __init__.py
    - Add the missing re-export
    - Re-run the failing test

    Also verify the sidecar import works:
    `python -c "from opendeviationbar.sidecar.checkpoint import *; print('sidecar checkpoint import OK')"`

  </action>
  <verify>
    <automated>cd /Users/terryli/eon/opendeviationbar-py && uv run --python 3.13 pytest tests/ -x -q --timeout=60 2>&1 | tail -10</automated>
  </verify>
  <acceptance_criteria>
    - `uv run --python 3.13 pytest tests/ -x -q --timeout=60` exits 0 with all tests passing
    - `python -c "from opendeviationbar.sidecar.checkpoint import *"` exits 0
  </acceptance_criteria>
  <done>All Python tests pass, zero regressions from checkpoint package conversion</done>
</task>

</tasks>

<verification>
1. `python -c "from opendeviationbar.checkpoint import populate_cache_resumable, populate_cache_multi_threshold, PopulationCheckpoint, _checkpoint_provenance"` -- imports work
2. `python -c "from opendeviationbar import populate_cache_resumable, populate_cache_multi_threshold"` -- package-level re-exports work
3. `uv run --python 3.13 pytest tests/ -x -q --timeout=60` -- all tests pass
4. `ls python/opendeviationbar/checkpoint/` shows __init__.py, __init__.pyi, models.py, storage.py, engine.py
5. `! test -f python/opendeviationbar/checkpoint.py` -- old file gone (it's now a directory)
</verification>

<success_criteria>

- checkpoint.py replaced by checkpoint/ package with models.py, storage.py, engine.py, **init**.py, **init**.pyi
- All public symbols importable from opendeviationbar.checkpoint
- All Python tests pass unchanged
- sidecar/checkpoint.py import of \_checkpoint_provenance still works
  </success_criteria>

<output>
After completion, create `.planning/phases/01-modularize-long-files-live-engine-rs-checkpoint-py-backfill-py/01-02-SUMMARY.md`
</output>
