---
phase: 05-seeder-rest-ratelimit
plan: 02
subsystem: infra
tags: [rate-limiting, pyo3, systemd, cross-process, binance-api]

# Dependency graph
requires:
  - phase: 05-01
    provides: "Seeder script with REST_CEILING=800, write_ticks() dedup gate"
provides:
  - "RateLimitCoordinator for cross-process REST weight reporting"
  - "PyAdaptiveRateLimiter PyO3 class for Python-side rate limit access"
  - "Per-process REST ceiling env vars in all systemd services"
  - "Kintsugi backfill rate limit awareness before REST calls"
affects: [05-03, 05-04]

# Tech tracking
tech-stack:
  added: []
  patterns:
    [
      "file-based cross-process coordination via atomic JSON rename",
      "utilization check before REST fetch",
    ]

key-files:
  created:
    - python/opendeviationbar/ratelimit/__init__.py
    - python/opendeviationbar/ratelimit/file_coordinator.py
    - tests/test_ratelimit_coordinator.py
  modified:
    - src/binance_bindings.rs
    - src/lib.rs
    - python/opendeviationbar/backfill.py
    - scripts/systemd/opendeviationbar-sidecar.service
    - scripts/systemd/opendeviationbar-kintsugi.service
    - scripts/systemd/opendeviationbar-kintsugi-catchup.service

key-decisions:
  - "PyAdaptiveRateLimiter.acquire() uses inline tokio runtime (no py.allow_threads) matching existing fetch pattern"
  - "Rate limit check added to both _fetch_aggtrades_rest and _fetch_aggtrades_by_id as single gate function"

patterns-established:
  - "Cross-process rate limit coordination: atomic JSON file at /tmp/opendeviationbar-ratelimit.json"
  - "Budget allocation D-13: sidecar=2400, kintsugi=2000, seeder=800, reserve=800"

requirements-completed: [SEEDER-02, SEEDER-03]

# Metrics
duration: 6min
completed: 2026-03-22
---

# Phase 05 Plan 02: Cross-Process REST Rate Limit Coordination Summary

**File-based RateLimitCoordinator + PyAdaptiveRateLimiter PyO3 binding + kintsugi rate limit gating + per-process systemd ceiling env vars**

## Performance

- **Duration:** 6 min
- **Started:** 2026-03-22T23:08:32Z
- **Completed:** 2026-03-22T23:15:13Z
- **Tasks:** 3
- **Files modified:** 9

## Accomplishments

- RateLimitCoordinator with atomic JSON state sharing, stale entry expiry, and budget constants matching D-13
- PyAdaptiveRateLimiter PyO3 class exposing acquire(), utilization(), remaining() to Python (D-16)
- Kintsugi backfill checks REST utilization before every fetch, sleeping when >85% to prevent retry storms (D-17)
- All systemd services configured with per-process REST ceilings: sidecar=2400, kintsugi=2000, kintsugi-catchup=2000

## Task Commits

Each task was committed atomically:

1. **Task 1: RateLimitCoordinator with file-based cross-process state sharing** - `43fe4af` (feat, TDD)
2. **Task 2: Per-process REST ceiling env vars in systemd services** - `de3d57b` (chore)
3. **Task 3: PyAdaptiveRateLimiter PyO3 + kintsugi REST rate limit wrapping** - `4f5ce67` (feat)

## Files Created/Modified

- `python/opendeviationbar/ratelimit/__init__.py` - Package init with public exports
- `python/opendeviationbar/ratelimit/file_coordinator.py` - RateLimitCoordinator class with atomic JSON state sharing
- `tests/test_ratelimit_coordinator.py` - 10 tests covering all coordinator behaviors
- `src/binance_bindings.rs` - PyAdaptiveRateLimiter pyclass with acquire/utilization/remaining
- `src/lib.rs` - Registered PyAdaptiveRateLimiter in module init
- `python/opendeviationbar/backfill.py` - \_check_rate_limit_before_rest() gate function
- `scripts/systemd/opendeviationbar-sidecar.service` - REST_CEILING=2400
- `scripts/systemd/opendeviationbar-kintsugi.service` - REST_CEILING=2000
- `scripts/systemd/opendeviationbar-kintsugi-catchup.service` - REST_CEILING=2000

## Decisions Made

- PyAdaptiveRateLimiter.acquire() uses inline tokio::runtime::Runtime::new() without py.allow_threads -- matches the pattern used by all other fetch functions in binance_bindings.rs
- Rate limit utilization check added as a single helper function \_check_rate_limit_before_rest() called by both_fetch_aggtrades_rest and \_fetch_aggtrades_by_id, covering all downstream callers

## Deviations from Plan

### Auto-fixed Issues

**1. [Rule 1 - Bug] Fixed PyO3 py.allow_threads compile error**

- **Found during:** Task 3 (PyAdaptiveRateLimiter implementation)
- **Issue:** `py.allow_threads(|| { ... })` with PyResult return type failed to compile in PyO3 0.28
- **Fix:** Removed py.allow_threads wrapper, used direct rt.block_on() matching existing pattern in the file
- **Files modified:** src/binance_bindings.rs
- **Verification:** maturin develop builds successfully
- **Committed in:** 4f5ce67

---

**Total deviations:** 1 auto-fixed (1 bug)
**Impact on plan:** Minimal -- adjusted PyO3 API call pattern to match existing codebase convention.

## Issues Encountered

None beyond the auto-fixed deviation above.

## User Setup Required

None - no external service configuration required.

## Known Stubs

None - all data sources are wired and functional.

## Next Phase Readiness

- Cross-process rate limit infrastructure complete
- Plans 03-04 can build on RateLimitCoordinator for seeder integration and heartbeat telemetry

---

_Phase: 05-seeder-rest-ratelimit_
_Completed: 2026-03-22_
