---
phase: 29-delete-strategy-exchange-gap-floor
plan: 01
type: execute
wave: 1
depends_on: []
files_modified:
  - python/opendeviationbar/sidecar/__init__.py
  - tests/test_sidecar_delete_strategy.py
autonomous: true
requirements: [RESUME-04]

must_haves:
  truths:
    - "Sidecar Aion DELETE phase completes in under 60 seconds for 42 partitions"
    - "No-op partitions (max_tid <= resume_tid) are skipped entirely — zero mutations dispatched"
    - "Extra bars are preferred over gaps — DELETE only removes bars AFTER resume_tid, never before"
    - "DELETE timings are logged per-symbol with skip counts for observability"
  artifacts:
    - path: "python/opendeviationbar/sidecar/__init__.py"
      provides: "Optimized Aion DELETE strategy with skip-if-clean check"
      contains: "_aion_delete_after_resume"
    - path: "tests/test_sidecar_delete_strategy.py"
      provides: "Unit tests for DELETE strategy skip logic"
      contains: "test_skip_noop_partitions"
  key_links:
    - from: "python/opendeviationbar/sidecar/__init__.py"
      to: "opendeviationbar_cache.open_deviation_bars"
      via: "max(last_agg_trade_id) pre-check query then DELETE IN PARTITION"
      pattern: "max\\(last_agg_trade_id\\)"
---

<objective>
Optimize the Aion sidecar DELETE strategy from ~6 minutes (42 partitions x ~8.5s each with mutations_sync=1) to under 60 seconds.

Purpose: Sidecar restart currently takes ~6 minutes just for the DELETE phase because it issues 42 synchronous DELETE IN PARTITION mutations regardless of whether there are any bars to delete. Most restarts have 0 bars after resume_tid in most partitions — the DELETE is a no-op but still triggers a full partition scan. By pre-checking max(last_agg_trade_id) per partition and skipping no-ops, we eliminate ~90% of mutations.

Output: Optimized `_aion_delete_after_resume()` function in sidecar/**init**.py with unit tests.
</objective>

<execution_context>
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@python/opendeviationbar/sidecar/__init__.py
@python/opendeviationbar/sidecar/CLAUDE.md

<interfaces>
<!-- Current Aion DELETE code in sidecar/__init__.py (lines 342-363) -->
```python
# Step 3: DELETE bars after resume_tid (not date-based — Aion has no days).
# Removes any bars from crashed/previous sessions beyond the resume point.
t_del_start = time.monotonic()
del_count = 0
for sym in aion_symbols:
    resume_tid = resume_tids.get(sym)
    if resume_tid is None:
        continue
    for thr in get_thresholds_for_symbol(sym):
        ch_cache.client.command(
            "ALTER TABLE opendeviationbar_cache.open_deviation_bars "
            f"DELETE IN PARTITION ('{sym}', {thr}) "
            f"WHERE first_agg_trade_id > {resume_tid} "
            "SETTINGS mutations_sync = 1"
        )
        del_count += 1
t_del_elapsed = time.monotonic() - t_del_start
logger.info(
    "aion: DELETE after resume_tid — %d partitions in %.1fs",
    del_count,
    t_del_elapsed,
)
```

<!-- kintsugi_log schema for exchange gaps (from schema.sql) -->

```sql
CREATE TABLE IF NOT EXISTS opendeviationbar_cache.kintsugi_log (
    timestamp DateTime64(3, 'UTC') DEFAULT now64(3, 'UTC'),
    symbol LowCardinality(String),
    threshold_decimal_bps UInt32,
    gap_start_us Int64,
    gap_end_us Int64,
    gap_hours Float64,
    action LowCardinality(String),
    result LowCardinality(String),  -- 'exchange_gap' for dismissed gaps
    ariadne_from_id Int64 DEFAULT 0,
    ouroboros_mode LowCardinality(String) DEFAULT 'day'
)
```

<!-- Partition key from CLAUDE.md section 7 -->
<!-- PARTITION BY (symbol, threshold_decimal_bps) — 64-72 partitions total -->
</interfaces>
</context>

<tasks>

<task type="auto" tdd="true">
  <name>Task 1: Implement skip-if-clean DELETE strategy with pre-check query</name>
  <files>python/opendeviationbar/sidecar/__init__.py, tests/test_sidecar_delete_strategy.py</files>
  <read_first>
    - python/opendeviationbar/sidecar/__init__.py (lines 260-375 — full Aion startup path)
    - python/opendeviationbar/sidecar/CLAUDE.md
    - python/opendeviationbar/clickhouse/CLAUDE.md (DELETE strategies section)
  </read_first>
  <behavior>
    - Test 1: test_skip_noop_partitions — When max(last_agg_trade_id) <= resume_tid for a partition, zero DELETE commands are issued for that partition
    - Test 2: test_delete_only_needed_partitions — When 2 of 4 partitions have bars after resume_tid, exactly 2 DELETE commands are issued
    - Test 3: test_all_partitions_clean_skips_all — When all partitions are clean (max_tid <= resume_tid), zero DELETE commands are issued and elapsed time is ~0
    - Test 4: test_logging_includes_skip_count — Log message includes both deleted and skipped partition counts
    - Test 5: test_resume_tid_none_skips_symbol — Symbols with resume_tid=None skip the entire DELETE check
  </behavior>
  <action>
    Extract the Aion DELETE logic (currently lines 342-363 in sidecar/__init__.py) into a standalone function `_aion_delete_after_resume(ch_cache, aion_symbols, resume_tids, get_thresholds_fn)` that:

    1. **Batch pre-check query**: For each symbol with a resume_tid, run a SINGLE query that gets max(last_agg_trade_id) per threshold_decimal_bps partition:
       ```sql
       SELECT threshold_decimal_bps, max(last_agg_trade_id) AS max_tid
       FROM opendeviationbar_cache.open_deviation_bars FINAL
       WHERE symbol = '{sym}' AND first_agg_trade_id > 0
       GROUP BY threshold_decimal_bps
       ```
       This is ONE query per symbol (not per partition), returning all thresholds at once.

    2. **Skip no-op partitions**: Only issue `DELETE IN PARTITION` for partitions where `max_tid > resume_tid`. Skip the rest entirely.

    3. **Keep mutations_sync=1**: Synchronous DELETE is non-negotiable (per CLAUDE.md). The speedup comes from skipping no-ops, not from async mutations.

    4. **Log with observability**: Log format:
       ```
       aion: DELETE after resume_tid — {deleted}/{total} partitions mutated ({skipped} skipped) in {elapsed:.1f}s
       ```

    5. **Replace inline code**: Replace the current inline DELETE loop (lines 342-363) with a call to `_aion_delete_after_resume()`.

    Write tests in `tests/test_sidecar_delete_strategy.py` using unittest.mock to mock `ch_cache.client.command()` and `ch_cache.client.query()`. Verify skip logic without requiring a real ClickHouse connection.

    IMPORTANT: The pre-check query uses `FINAL` — same as the existing resume TID query on line 291. This is correct because we need merged results (not duplicate rows from different parts).

  </action>
  <verify>
    <automated>cd /Users/terryli/eon/opendeviationbar-py && uv run --python 3.13 pytest tests/test_sidecar_delete_strategy.py -xvs 2>&1 | tail -30</automated>
  </verify>
  <acceptance_criteria>
    - grep -n "_aion_delete_after_resume" python/opendeviationbar/sidecar/__init__.py shows function definition AND call site
    - grep -n "skipped" python/opendeviationbar/sidecar/__init__.py shows skip count in log message
    - grep -n "max_tid > resume_tid\|max_tid >" python/opendeviationbar/sidecar/__init__.py shows the skip condition
    - grep -n "test_skip_noop" tests/test_sidecar_delete_strategy.py shows test exists
    - The old inline DELETE loop (bare `for thr in get_thresholds_for_symbol` + `DELETE IN PARTITION` without pre-check) is GONE
  </acceptance_criteria>
  <done>
    - _aion_delete_after_resume() function exists and is called from _create_engine()
    - Pre-check query runs once per symbol (not per partition)
    - Skip logic prevents no-op DELETE IN PARTITION commands
    - Log message shows deleted/total/skipped counts
    - All 5 test cases pass
  </done>
</task>

</tasks>

<verification>
- `uv run --python 3.13 pytest tests/test_sidecar_delete_strategy.py -xvs` passes all tests
- `grep -c "DELETE IN PARTITION" python/opendeviationbar/sidecar/__init__.py` shows DELETE is still used (not removed, just gated)
- The old inline loop without pre-check is gone — replaced by `_aion_delete_after_resume()` call
</verification>

<success_criteria>

- DELETE strategy uses skip-if-clean pre-check (batch query per symbol, skip no-op partitions)
- mutations_sync=1 preserved on all DELETE commands
- Function is extracted and testable independently
- Observability: skip count logged alongside mutation count and elapsed time
  </success_criteria>

<output>
After completion, create `.planning/phases/29-delete-strategy-exchange-gap-floor/29-01-SUMMARY.md`
</output>
