---
phase: 05-seeder-rest-ratelimit
plan: 01
subsystem: storage
tags: [parquet, trade-id, validation, systemd, seeder, dedup]

# Dependency graph
requires: []
provides:
  - validate_trade_id_continuity() function in parquet_io.py
  - Seeder write_ticks() routing with dedup and continuity validation
  - systemd Conflicts= directive preventing concurrent seeder runs
  - Per-process REST ceiling env var (OPENDEVIATIONBAR_REST_CEILING)
affects: [05-02, 05-03, 05-04]

# Tech tracking
tech-stack:
  added: []
  patterns:
    [
      trade-ID continuity validation before Parquet writes,
      write_ticks routing for all tick consumers,
    ]

key-files:
  created:
    - tests/test_parquet_validation.py
  modified:
    - python/opendeviationbar/storage/parquet_io.py
    - python/opendeviationbar/storage/parquet.py
    - scripts/tick_cache_seeder.py
    - scripts/systemd/opendeviationbar-seeder.service

key-decisions:
  - "Validation integrated at three write paths in write_ticks() (fresh, corrupted-rewrite, post-merge) for comprehensive coverage"
  - "Seeder routes through write_ticks() which handles dedup, normalization, and continuity validation — single responsibility"

patterns-established:
  - "All Parquet tick writes must go through write_ticks() to get dedup + continuity validation"
  - "validate_trade_id_continuity() sorts before checking, so callers need not pre-sort"

requirements-completed: [SEEDER-04]

# Metrics
duration: 4min
completed: 2026-03-22
---

# Phase 05 Plan 01: Parquet Write Hardening Summary

**Trade-ID continuity validation in write_ticks(), seeder routed through dedup path, systemd overlap prevention via Conflicts=**

## Performance

- **Duration:** 4 min
- **Started:** 2026-03-22T22:59:15Z
- **Completed:** 2026-03-22T23:03:15Z
- **Tasks:** 2
- **Files modified:** 5

## Accomplishments

- Added `validate_trade_id_continuity()` to parquet_io.py: sorts by agg_trade_id, verifies all consecutive diffs == 1
- Integrated validation into all three write paths in `write_ticks()` (fresh write, corrupted-rewrite, post-merge)
- Replaced seeder's direct `_atomic_write_parquet()` call with `storage.write_ticks()` for automatic dedup + validation
- Added systemd `Conflicts=` directive and `OPENDEVIATIONBAR_REST_CEILING=800` env var to seeder service

## Task Commits

Each task was committed atomically:

1. **Task 1 (RED): Failing tests** - `2269472` (test)
2. **Task 1 (GREEN): Implementation** - `d085766` (feat)
3. **Task 2: systemd Conflicts= and REST ceiling** - `8bddfaa` (chore)

## Files Created/Modified

- `python/opendeviationbar/storage/parquet_io.py` - Added validate_trade_id_continuity() function
- `python/opendeviationbar/storage/parquet.py` - Integrated continuity validation into write_ticks() at three write paths
- `scripts/tick_cache_seeder.py` - Replaced \_atomic_write_parquet() with storage.write_ticks()
- `scripts/systemd/opendeviationbar-seeder.service` - Added Conflicts= and REST_CEILING env var
- `tests/test_parquet_validation.py` - 7 test cases for trade-ID continuity validation

## Decisions Made

- Validation integrated at three write paths in write_ticks() (fresh, corrupted-rewrite, post-merge) for comprehensive coverage
- Seeder routes through write_ticks() which handles dedup, normalization, and continuity validation -- single responsibility

## Deviations from Plan

None - plan executed exactly as written.

## Issues Encountered

None

## User Setup Required

None - no external service configuration required.

## Known Stubs

None - all functionality is fully wired.

## Next Phase Readiness

- Trade-ID continuity validation and dedup write path are in place for Plan 02 (REST fallback rate limiting)
- REST ceiling env var is set in systemd, ready for Plan 02's AdaptiveRateLimiter integration

---

_Phase: 05-seeder-rest-ratelimit_
_Completed: 2026-03-22_
