---
phase: 02-kintsugi-yesterday-priority-queue
plan: 01
type: execute
wave: 1
depends_on: []
files_modified:
  - python/opendeviationbar/sidecar/yesterday_signal.py
  - python/opendeviationbar/kintsugi/orchestrator.py
  - tests/test_yesterday_priority_queue.py
  - tests/test_yesterday_integrity.py
autonomous: true
requirements: [STARTUP-03]

must_haves:
  truths:
    - "Kintsugi reads yesterday-repair signal file at start of each pass"
    - "Flagged symbols are repaired before normal shard discovery results"
    - "Signal file is deleted after all flagged shards are processed"
    - "Non-flagged symbols still follow normal recency-first ordering"
    - "Stale signal files (>24h) are discarded with warning"
  artifacts:
    - path: "python/opendeviationbar/sidecar/yesterday_signal.py"
      provides: "clear_yesterday_signal() function"
      contains: "def clear_yesterday_signal"
    - path: "python/opendeviationbar/kintsugi/orchestrator.py"
      provides: "Yesterday priority queue injection in kintsugi_pass"
      contains: "_load_yesterday_priority_shards"
    - path: "tests/test_yesterday_priority_queue.py"
      provides: "Tests for signal consumption, shard creation, queue ordering, cleanup"
      min_lines: 80
  key_links:
    - from: "python/opendeviationbar/kintsugi/orchestrator.py"
      to: "python/opendeviationbar/sidecar/yesterday_signal.py"
      via: "read_yesterday_signal + clear_yesterday_signal imports"
      pattern: "from opendeviationbar\\.sidecar\\.yesterday_signal import"
    - from: "python/opendeviationbar/kintsugi/orchestrator.py"
      to: "python/opendeviationbar/symbol_registry"
      via: "get_thresholds_for_symbol for expanding symbols to (symbol, threshold) pairs"
      pattern: "get_thresholds_for_symbol"
---

<objective>
Kintsugi consumes yesterday-repair signal file and repairs flagged symbols with priority before normal shard discovery.

Purpose: Complete the sidecar-to-kintsugi handoff so broken yesterday data is repaired within the first kintsugi pass (<5 min at 24 workers), eliminating the 4h blocking repair from sidecar startup.

Output: Modified orchestrator.py with signal consumption + priority injection, clear_yesterday_signal() in yesterday_signal.py, comprehensive tests.
</objective>

<execution_context>
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/01-yesterday-integrity-check-signal-file/01-01-SUMMARY.md

@python/opendeviationbar/sidecar/yesterday_signal.py
@python/opendeviationbar/kintsugi/orchestrator.py
@python/opendeviationbar/kintsugi/discovery.py
@python/opendeviationbar/kintsugi/daemon.py

<interfaces>
<!-- Key types and contracts the executor needs. -->

From python/opendeviationbar/sidecar/yesterday_signal.py:

```python
SIGNAL_FILE_PATH = "/var/tmp/opendeviationbar-yesterday-repair.json"

def write_yesterday_signal(flagged: dict[str, str], sidecar_pid: int | None = None) -> None: ...
def read_yesterday_signal() -> dict | None: ...
# Signal format: {"symbols": {"SYM": "reason"}, "timestamp": "ISO", "sidecar_pid": N}
```

From python/opendeviationbar/kintsugi/discovery.py:

```python
@dataclass
class Shard:
    symbol: str
    threshold_decimal_bps: int
    gap_start_ms: int
    gap_end_ms: int
    gap_hours: float
    ouroboros_boundaries: list[int] = field(default_factory=list)
    anchor_last_tid: int | None = None
    resume_first_tid: int | None = None
    missing_trade_count: int = 0
    is_overlap: bool = False
    is_genesis: bool = False

def _shard_sort_key(shard: Shard) -> tuple[int, int, int]: ...
def discover_shards(*, symbol: str | None = None, threshold: int | None = None) -> list[Shard]: ...
```

From python/opendeviationbar/symbol_registry.py:

```python
def get_thresholds_for_symbol(symbol: str) -> tuple[int, ...]: ...
def get_registered_symbols(enabled_only: bool = False) -> list[str]: ...
```

</interfaces>
</context>

<tasks>

<task type="auto" tdd="true">
  <name>Task 1: Add clear_yesterday_signal + _load_yesterday_priority_shards</name>
  <files>
    python/opendeviationbar/sidecar/yesterday_signal.py,
    python/opendeviationbar/kintsugi/orchestrator.py,
    tests/test_yesterday_priority_queue.py,
    tests/test_yesterday_integrity.py
  </files>
  <behavior>
    - Test: clear_yesterday_signal deletes signal file when it exists
    - Test: clear_yesterday_signal is no-op when file does not exist (no exception)
    - Test: _load_yesterday_priority_shards returns empty list when no signal file
    - Test: _load_yesterday_priority_shards returns Shard objects for each (symbol, threshold) pair from signal
    - Test: _load_yesterday_priority_shards creates shards with yesterday's date range (gap_start_ms=yesterday 00:00 UTC, gap_end_ms=yesterday 23:59:59.999 UTC)
    - Test: _load_yesterday_priority_shards discards signal files older than 24h with WARNING log
    - Test: _load_yesterday_priority_shards skips symbols not in registry (enabled_only=True)
    - Test: returned shards have is_overlap=False, is_genesis=False, anchor_last_tid=None, gap_hours=24.0
  </behavior>
  <action>
    1. Add `clear_yesterday_signal()` to `python/opendeviationbar/sidecar/yesterday_signal.py`:
       - Deletes SIGNAL_FILE_PATH if it exists
       - Catches FileNotFoundError silently (race-safe)
       - Logs INFO on successful deletion

    2. Add `_load_yesterday_priority_shards()` as a module-level function in `python/opendeviationbar/kintsugi/orchestrator.py` (above `kintsugi_pass`):
       ```python
       def _load_yesterday_priority_shards() -> list[Shard]:
       ```
       - Calls `read_yesterday_signal()` from `opendeviationbar.sidecar.yesterday_signal`
       - If None, returns []
       - Parses `timestamp` field; if older than 24h, log WARNING "stale yesterday-repair signal (age=Xh), discarding", call `clear_yesterday_signal()`, return []
       - For each symbol in `signal["symbols"]`:
         - Skip if symbol not in `get_registered_symbols(enabled_only=True)`
         - Get all thresholds via `get_thresholds_for_symbol(symbol)`
         - Create Shard for each threshold with:
           - `gap_start_ms` = yesterday midnight UTC (ms)
           - `gap_end_ms` = yesterday 23:59:59.999 UTC (ms)
           - `gap_hours` = 24.0
           - `anchor_last_tid` = None (kintsugi repair handles anchor discovery)
           - `resume_first_tid` = None
           - `missing_trade_count` = 0
           - `is_overlap` = False
           - `is_genesis` = False
       - Logs INFO "yesterday-repair signal: loaded N shards for M symbols (reasons: ...)"
       - Returns list of Shard objects

    3. Write tests to `tests/test_yesterday_priority_queue.py` covering all behaviors above.
       - Add `clear_yesterday_signal` tests to `tests/test_yesterday_integrity.py` (existing signal file test module).

    Import additions to orchestrator.py (at top, with existing imports):
    ```python
    from opendeviationbar.sidecar.yesterday_signal import (
        clear_yesterday_signal,
        read_yesterday_signal,
    )
    ```

  </action>
  <verify>
    <automated>cd /Users/terryli/eon/opendeviationbar-py && uv run --python 3.13 pytest tests/test_yesterday_priority_queue.py tests/test_yesterday_integrity.py -x -v 2>&1 | tail -30</automated>
  </verify>
  <done>
    - clear_yesterday_signal() deletes file or no-ops safely
    - _load_yesterday_priority_shards() returns correct Shard objects from signal file
    - Stale signals (>24h) are discarded with warning
    - All new tests pass
  </done>
</task>

<task type="auto">
  <name>Task 2: Wire priority queue into kintsugi_pass + signal cleanup</name>
  <files>
    python/opendeviationbar/kintsugi/orchestrator.py,
    tests/test_yesterday_priority_queue.py
  </files>
  <action>
    Modify `kintsugi_pass()` in `orchestrator.py` to inject yesterday priority shards:

    1. **Before** `discover_shards()` call (currently line 71), insert:
       ```python
       # STARTUP-03: yesterday priority queue — signal file from sidecar
       yesterday_shards = _load_yesterday_priority_shards()
       ```

    2. **After** `discover_shards()` returns and before sorting (between lines 71-79), merge:
       ```python
       if yesterday_shards:
           # Deduplicate: remove any shards from normal discovery that match
           # a yesterday priority shard (same symbol + threshold + yesterday date)
           yesterday_keys = {
               (s.symbol, s.threshold_decimal_bps) for s in yesterday_shards
           }
           shards = [s for s in shards if (s.symbol, s.threshold_decimal_bps) not in yesterday_keys]
           # Prepend: yesterday shards go FIRST (before sort reorders normal shards)
           shards = yesterday_shards + shards
           logger.info(
               "yesterday-repair: injected %d priority shards for %s",
               len(yesterday_shards),
               sorted({s.symbol for s in yesterday_shards}),
           )
           _emit_telemetry(
               "yesterday_priority_injected",
               n_shards=len(yesterday_shards),
               symbols=sorted({s.symbol for s in yesterday_shards}),
           )
       ```

    3. **After** the summary section (after line ~345, the results summary), add signal file cleanup:
       ```python
       # STARTUP-03: clear signal file after processing yesterday priority shards
       if yesterday_shards:
           # Check if all yesterday shards were processed (healed or failed, not skipped)
           yesterday_keys_set = {
               (s.symbol, s.threshold_decimal_bps) for s in yesterday_shards
           }
           processed_keys = {
               (r.shard.symbol, r.shard.threshold_decimal_bps) for r in results
           }
           unprocessed = yesterday_keys_set - processed_keys
           if unprocessed:
               logger.warning(
                   "yesterday-repair: %d shards not processed this pass (budget?), "
                   "keeping signal file for next pass",
                   len(unprocessed),
               )
           else:
               clear_yesterday_signal()
               logger.info("yesterday-repair: signal file cleared after processing all flagged shards")
       ```
       This goes right before the hook/telegram notification block (before line ~359 `try: from opendeviationbar.hooks`).

    4. Handle the edge case where `shards` is empty after `discover_shards()` but `yesterday_shards` is non-empty:
       - The current early return at lines 73-76 (`if not shards: return []`) must be adjusted.
       - Change the flow: move the early-return check to AFTER merging yesterday_shards:
         ```python
         shards = discover_shards(threshold=threshold)
         # ... merge yesterday_shards ...
         if not shards:
             logger.info("kintsugi: no shards found -- cache is clean")
             # Still clear signal if yesterday shards were loaded but all clean
             if yesterday_shards:
                 clear_yesterday_signal()
             return []
         ```

    5. Add tests to `tests/test_yesterday_priority_queue.py`:
       - Test: kintsugi_pass with signal file injects shards before normal discovery
       - Test: kintsugi_pass clears signal file after all yesterday shards processed
       - Test: kintsugi_pass keeps signal file when budget exhausted (some yesterday shards unprocessed)
       - Test: kintsugi_pass with empty normal discovery but non-empty signal file still processes
       - Test: deduplication — normal discovery shard for same (symbol, threshold) is removed when yesterday shard exists

    Mock strategy for kintsugi_pass tests:
    - Mock `discover_shards` to return controlled shards
    - Mock `_load_yesterday_priority_shards` for some tests, use real function with monkeypatched signal file for others
    - Mock `repair_shard` to return successful KintsugiResult
    - Mock `verify_repair` to return True
    - Mock `_select_repair_candidates` to pass through all shards (or control budget)
    - Mock `_emit_telemetry`, `_sd_notify`, `_maybe_log_repair` as no-ops
    - Mock `clear_yesterday_signal` to track calls
    - Mock ClickHouse-dependent functions (`_get_dismissed_set`, `_get_rate_limited_set`)

  </action>
  <verify>
    <automated>cd /Users/terryli/eon/opendeviationbar-py && uv run --python 3.13 pytest tests/test_yesterday_priority_queue.py tests/test_yesterday_integrity.py -x -v 2>&1 | tail -40</automated>
  </verify>
  <done>
    - kintsugi_pass reads signal file, injects yesterday shards at HEAD of queue
    - Signal file cleared after all flagged shards processed
    - Signal file preserved when budget exhausted (partial processing)
    - Normal shard ordering unaffected for non-flagged shards
    - Deduplication prevents double-processing of same (symbol, threshold) pair
    - All tests pass including existing test_yesterday_integrity.py
  </done>
</task>

</tasks>

<verification>
1. All new + existing tests pass:
   ```bash
   uv run --python 3.13 pytest tests/test_yesterday_priority_queue.py tests/test_yesterday_integrity.py -x -v
   ```

1. No regressions in existing kintsugi tests:

   ```bash
   uv run --python 3.13 pytest tests/ -k "kintsugi" -x -v
   ```

2. Import check:

   ```bash
   uv run --python 3.13 python -c "from opendeviationbar.sidecar.yesterday_signal import clear_yesterday_signal; print('OK')"
   uv run --python 3.13 python -c "from opendeviationbar.kintsugi.orchestrator import _load_yesterday_priority_shards; print('OK')"
   ```

3. Signal flow integration check:

   ```bash
   uv run --python 3.13 python -c "
   from opendeviationbar.sidecar.yesterday_signal import write_yesterday_signal, read_yesterday_signal, clear_yesterday_signal
   write_yesterday_signal({'BTCUSDT': 'stathera_gap'})
   sig = read_yesterday_signal()
   assert sig is not None and 'BTCUSDT' in sig['symbols']
   clear_yesterday_signal()
   assert read_yesterday_signal() is None
   print('Signal flow OK')
   "
   ```

</verification>

<success_criteria>

- SC-03a: Signal file read at start of kintsugi_pass, flagged symbols injected at HEAD of repair queue
- SC-03b: Signal file consumed (deleted) after ALL flagged shards processed in a pass
- SC-03c: Non-flagged symbols still follow normal recency-first ordering (existing \_shard_sort_key unchanged)
- SC-03d: Stale signals (>24h) discarded gracefully
- SC-03e: Budget exhaustion preserves signal file for next pass retry
  </success_criteria>

<output>
After completion, create `.planning/phases/02-kintsugi-yesterday-priority-queue/02-01-SUMMARY.md`
</output>
