---
phase: 01-modularize-long-files-live-engine-rs-checkpoint-py-backfill-py
plan: 03
subsystem: python-api
tags:
  [
    backfill,
    refactoring,
    modularization,
    memory-guard,
    rectification,
    vision-cache,
  ]

# Dependency graph
requires: []
provides:
  - "rectify.py -- bar rectification safety net (extracted from backfill.py)"
  - "memory_guard.py -- memory watermark + allocator purge utilities"
  - "vision_cache.py -- Redis-backed negative cache for Vision 404s"
  - "backfill.py trimmed to core gap-fill logic (~980 lines)"
affects: []

# Tech tracking
tech-stack:
  added: []
  patterns:
    - "Sibling module extraction with re-export for backward compatibility"

key-files:
  created:
    - "python/opendeviationbar/rectify.py"
    - "python/opendeviationbar/memory_guard.py"
    - "python/opendeviationbar/vision_cache.py"
  modified:
    - "python/opendeviationbar/backfill.py"

key-decisions:
  - "Re-export all extracted symbols from backfill.py for full backward compatibility"
  - "rectify.py imports _fetch_aggtrades_by_id_with_retry from backfill (not circular -- lazy import inside function)"
  - "memory_guard.py uses local import for BackfillConfig instead of _backfill_cfg() accessor"

patterns-established:
  - "Sibling extraction: move functions to new module, re-export from original for zero-breakage"

requirements-completed: []

# Metrics
duration: 10min
completed: 2026-03-23
---

# Phase 01 Plan 03: Backfill.py Extraction Summary

**Extracted 3 orthogonal concerns (rectification, memory guard, vision cache) from backfill.py into sibling modules, reducing it from 1423 to 980 lines with full backward compatibility**

## Performance

- **Duration:** 10 min
- **Started:** 2026-03-23T09:22:22Z
- **Completed:** 2026-03-23T09:32:35Z
- **Tasks:** 2
- **Files modified:** 4

## Accomplishments

- Extracted bar rectification (331 lines) into rectify.py with 4 functions
- Extracted memory watermark + allocator purge (71 lines) into memory_guard.py
- Extracted Vision negative cache (59 lines) into vision_cache.py
- All backward-compatible imports preserved via re-exports in backfill.py
- All 104 Python tests pass (3 pre-existing failures excluded)

## Task Commits

Each task was committed atomically:

1. **Task 1: Extract rectify.py, memory_guard.py, and vision_cache.py** - `ce3d44a` (refactor)
2. **Task 2: Run tests + fix missing re-exports** - `c2af5f8` (fix)

## Files Created/Modified

- `python/opendeviationbar/rectify.py` - Bar rectification safety net (rectify_recent_bars, \_delete_ghost_bar,\_fetch_trades_for_bar,\_emit_rectification_event)
- `python/opendeviationbar/memory_guard.py` - Memory watermark and allocator purge utilities (\_force_allocator_purge,\_check_memory_watermark)
- `python/opendeviationbar/vision_cache.py` - Redis-backed negative cache for Vision 404 responses (\_is_vision_cached_unavailable, \_mark_vision_unavailable)
- `python/opendeviationbar/backfill.py` - Trimmed from 1423 to 980 lines, re-exports all extracted symbols

## Decisions Made

- Re-export all extracted symbols (including \_WATERMARK_GC_RATIO,\_WATERMARK_ABORT_RATIO constants) from backfill.py for zero-breakage backward compatibility
- rectify.py uses lazy imports from backfill for \_fetch_aggtrades_by_id_with_retry (avoids circular import)
- memory_guard.py directly imports BackfillConfig instead of using_backfill_cfg() accessor (module is standalone)

## Deviations from Plan

### Auto-fixed Issues

**1. [Rule 1 - Bug] Missing _WATERMARK_\*\_RATIO re-exports**

- **Found during:** Task 2 (test suite verification)
- **Issue:** test_adaptive_sleep.py imports \_WATERMARK_ABORT_RATIO and_WATERMARK_GC_RATIO from backfill, but these constants were moved to memory_guard.py without re-export
- **Fix:** Added re-exports for \_WATERMARK_GC_RATIO and_WATERMARK_ABORT_RATIO in backfill.py imports from memory_guard
- **Files modified:** python/opendeviationbar/backfill.py
- **Verification:** All backfill-related tests pass
- **Committed in:** c2af5f8 (Task 2 commit)

---

**Total deviations:** 1 auto-fixed (1 bug)
**Impact on plan:** Essential for backward compatibility. No scope creep.

## Issues Encountered

- 3 pre-existing test failures (test_midnight_preflight, test_oracle_regression, test_yesterday_priority_queue) unrelated to this plan's changes -- confirmed by running tests before and after changes

## Known Stubs

None.

## User Setup Required

None - no external service configuration required.

## Next Phase Readiness

- backfill.py extraction complete, all 3 sibling modules functional
- Combined with plans 01-01 (live_engine.rs) and 01-02 (checkpoint.py), phase 01 modularization is complete

---

_Phase: 01-modularize-long-files-live-engine-rs-checkpoint-py-backfill-py_
_Completed: 2026-03-23_
