---
phase: 05-seeder-rest-ratelimit
plan: 01
type: execute
wave: 1
depends_on: []
files_modified:
  - python/opendeviationbar/storage/parquet_io.py
  - python/opendeviationbar/storage/parquet.py
  - scripts/tick_cache_seeder.py
  - scripts/systemd/opendeviationbar-seeder.service
  - tests/test_parquet_validation.py
autonomous: true
requirements: [SEEDER-04]

must_haves:
  truths:
    - "Every Parquet tick write validates trade-ID continuity before committing"
    - "Seeder routes all writes through write_ticks() with dedup instead of direct _atomic_write_parquet()"
    - "Concurrent seeder runs are prevented by systemd Conflicts= directive"
  artifacts:
    - path: "python/opendeviationbar/storage/parquet_io.py"
      provides: "validate_trade_id_continuity() function"
      contains: "def validate_trade_id_continuity"
    - path: "scripts/tick_cache_seeder.py"
      provides: "Seeder using storage.write_ticks() instead of _atomic_write_parquet()"
    - path: "scripts/systemd/opendeviationbar-seeder.service"
      provides: "Conflicts= directive preventing concurrent runs"
      contains: "Conflicts=opendeviationbar-seeder.service"
    - path: "tests/test_parquet_validation.py"
      provides: "Tests for trade-ID continuity validation"
  key_links:
    - from: "scripts/tick_cache_seeder.py"
      to: "python/opendeviationbar/storage/parquet.py"
      via: "storage.write_ticks() call replacing _atomic_write_parquet()"
      pattern: "storage\\.write_ticks"
    - from: "python/opendeviationbar/storage/parquet.py"
      to: "python/opendeviationbar/storage/parquet_io.py"
      via: "validate_trade_id_continuity() called before write"
      pattern: "validate_trade_id_continuity"
---

<objective>
Harden Parquet tick writes with trade-ID continuity validation, route seeder through write_ticks() dedup path, and add systemd overlap prevention.

Purpose: Ensures Parquet tick files have zero trade-ID gaps (D-07), eliminates duplicate trades from concurrent/repeated seeder runs (D-09), and prevents systemd timer overlap (D-05). These are foundational safety nets that all subsequent REST fallback work depends on.

Output: validate_trade_id_continuity() function, seeder write_ticks() routing, systemd Conflicts= directive, test suite.
</objective>

<execution_context>
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/05-seeder-rest-ratelimit/05-CONTEXT.md

@python/opendeviationbar/storage/parquet_io.py
@python/opendeviationbar/storage/parquet.py
@scripts/tick_cache_seeder.py
@scripts/systemd/opendeviationbar-seeder.service
</context>

<tasks>

<task type="auto" tdd="true">
  <name>Task 1: Add trade-ID continuity validation to Parquet I/O and route seeder through write_ticks()</name>
  <files>python/opendeviationbar/storage/parquet_io.py, python/opendeviationbar/storage/parquet.py, scripts/tick_cache_seeder.py, tests/test_parquet_validation.py</files>
  <read_first>
    - python/opendeviationbar/storage/parquet_io.py (current validation functions, understand existing pattern)
    - python/opendeviationbar/storage/parquet.py (write_ticks() implementation with dedup, lines 188-230+)
    - scripts/tick_cache_seeder.py (lines 294-396, _seed_symbol() with direct _atomic_write_parquet() call at line 360)
  </read_first>
  <behavior>
    - Test 1: validate_trade_id_continuity() returns True for df with consecutive agg_trade_ids [100, 101, 102, 103]
    - Test 2: validate_trade_id_continuity() returns False for df with gap [100, 101, 103, 104] (missing 102)
    - Test 3: validate_trade_id_continuity() returns False for df with duplicate [100, 101, 101, 102]
    - Test 4: validate_trade_id_continuity() returns True for single-row df (trivially valid)
    - Test 5: validate_trade_id_continuity() returns True for empty df (trivially valid)
    - Test 6: validate_trade_id_continuity() returns True for unsorted input (function sorts before checking)
  </behavior>
  <action>
    1. In `python/opendeviationbar/storage/parquet_io.py`, add function `validate_trade_id_continuity(df: pl.DataFrame) -> bool` (per D-07):
       - If df has fewer than 2 rows or "agg_trade_id" not in columns, return True
       - Sort by "agg_trade_id"
       - Compute diffs: `sorted_df["agg_trade_id"].diff().drop_nulls()`
       - Return `(deltas == 1).all()` -- every consecutive pair increments by exactly 1

    2. In `python/opendeviationbar/storage/parquet.py`, in `write_ticks()` method, after the dedup anti-join merge step and before the `_atomic_write_parquet()` call, add:
       ```python
       from .parquet_io import validate_trade_id_continuity
       if not validate_trade_id_continuity(final_df):
           logger.warning("%s %s: trade-ID continuity check failed, rejecting write", symbol, day_key)
           continue  # skip this day's write
       ```
       This ensures ALL consumers of write_ticks() get validation for free.

    3. In `scripts/tick_cache_seeder.py` `_seed_symbol()` function (line ~358-361), replace:
       ```python
       from opendeviationbar.storage.parquet_io import _atomic_write_parquet
       ...
       day_path = storage._get_parquet_path(cache_sym, date_str)
       _atomic_write_parquet(df, day_path)
       ```
       With:
       ```python
       storage.write_ticks(cache_sym, df)
       ```
       Remove the import of `_atomic_write_parquet` from the function (line 311). The `storage.write_ticks()` call handles dedup, continuity validation, atomic write, and timestamp normalization (per D-09).

    4. Create `tests/test_parquet_validation.py` with the 6 test cases defined in `<behavior>` above. Use `polars.DataFrame` with columns ["agg_trade_id", "price", "quantity", "timestamp", "is_buyer_maker", "first_trade_id", "last_trade_id"] to match the real schema.

  </action>
  <verify>
    <automated>cd /Users/terryli/eon/opendeviationbar-py && uv run --python 3.13 pytest tests/test_parquet_validation.py -x -v</automated>
  </verify>
  <acceptance_criteria>
    - parquet_io.py contains `def validate_trade_id_continuity(df: pl.DataFrame) -> bool`
    - parquet_io.py contains `(deltas == 1).all()` or equivalent polars check
    - parquet.py write_ticks() contains `validate_trade_id_continuity` call
    - tick_cache_seeder.py _seed_symbol() contains `storage.write_ticks(cache_sym, df)` instead of `_atomic_write_parquet(df, day_path)`
    - tick_cache_seeder.py does NOT import `_atomic_write_parquet` in _seed_symbol()
    - tests/test_parquet_validation.py exists with 6+ test functions
    - All tests pass
  </acceptance_criteria>
  <done>Trade-ID continuity validation integrated into write_ticks(), seeder routes through dedup path, all tests pass.</done>
</task>

<task type="auto">
  <name>Task 2: Add systemd Conflicts= directive to seeder service</name>
  <files>scripts/systemd/opendeviationbar-seeder.service</files>
  <read_first>
    - scripts/systemd/opendeviationbar-seeder.service (current service file, full contents)
    - scripts/systemd/opendeviationbar-kintsugi.service (reference for Conflicts= pattern)
  </read_first>
  <action>
    In `scripts/systemd/opendeviationbar-seeder.service`, add to the `[Unit]` section (per D-05):
    ```ini
    Conflicts=opendeviationbar-seeder.service
    ```
    This prevents concurrent seeder runs when a previous run exceeds the 5-minute timer interval. Same pattern already used by kintsugi-catchup (which has `Conflicts=opendeviationbar-kintsugi.service`).

    Also add the per-process rate limit ceiling env var to prepare for Plan 02 (per D-13):
    ```ini
    Environment=OPENDEVIATIONBAR_REST_CEILING=800
    ```
    This sets the seeder's per-process REST budget to 800 weight/min out of the 6000 IP-wide total.

  </action>
  <verify>
    <automated>grep -c "Conflicts=opendeviationbar-seeder.service" /Users/terryli/eon/opendeviationbar-py/scripts/systemd/opendeviationbar-seeder.service && grep -c "OPENDEVIATIONBAR_REST_CEILING=800" /Users/terryli/eon/opendeviationbar-py/scripts/systemd/opendeviationbar-seeder.service</automated>
  </verify>
  <acceptance_criteria>
    - opendeviationbar-seeder.service [Unit] section contains `Conflicts=opendeviationbar-seeder.service`
    - opendeviationbar-seeder.service contains `Environment=OPENDEVIATIONBAR_REST_CEILING=800`
  </acceptance_criteria>
  <done>Seeder systemd service prevents timer overlap via Conflicts= and has per-process REST ceiling configured.</done>
</task>

</tasks>

<verification>
- `uv run --python 3.13 pytest tests/test_parquet_validation.py -x -v` passes
- `grep "validate_trade_id_continuity" python/opendeviationbar/storage/parquet_io.py` finds the function
- `grep "storage.write_ticks" scripts/tick_cache_seeder.py` confirms seeder uses write_ticks()
- `grep "Conflicts=" scripts/systemd/opendeviationbar-seeder.service` confirms overlap prevention
</verification>

<success_criteria>

1. validate_trade_id_continuity() exists in parquet_io.py and correctly identifies gaps/duplicates
2. write_ticks() in parquet.py calls validate_trade_id_continuity() before writing
3. Seeder uses storage.write_ticks() instead of direct_atomic_write_parquet() (D-09)
4. systemd service has Conflicts= directive (D-05)
5. All test cases pass
   </success_criteria>

<output>
After completion, create `.planning/phases/05-seeder-rest-ratelimit/05-01-SUMMARY.md`
</output>
