---
phase: 18-monitor-validation-gates
verified: 2026-03-27T17:30:00Z
status: gaps_found
score: 5/8 must-haves verified
re_verification: false
gaps:
  - truth: "Day-mode symbols still get the existing crossover_tid orphan completion check"
    status: failed
    reason: "6 pre-existing TestYesterdayCompletion tests now fail because they use BTCUSDT (which is ouroboros_mode=aion in symbols.toml) without mocking get_symbol_entries. Phase 18 added mode-branching to collect_yesterday_completion(), which now calls get_symbol_entries() and routes BTCUSDT to the aion freshness path, making the day-mode assertions in the pre-existing tests unreachable."
    artifacts:
      - path: "tests/test_heartbeat_yesterday_junction.py"
        issue: "TestYesterdayCompletion class (tests 170-340) does not mock get_symbol_entries(); all 6 tests fail with 'no_bars' instead of expected day-mode results"
    missing:
      - "Add get_symbol_entries mock to each TestYesterdayCompletion test that sets BTCUSDT to ouroboros_mode='day' so day-mode crossover_tid check is exercised (same _make_symbol_entry pattern used by TestAionFreshness)"
  - truth: "MON-01 marked complete in REQUIREMENTS.md"
    status: failed
    reason: "REQUIREMENTS.md still shows MON-01 as [ ] (unchecked) despite Plan 02 SUMMARY claiming requirements-completed: [MON-01]. The implementation exists and is correct, but the requirements file was not updated."
    artifacts:
      - path: ".planning/REQUIREMENTS.md"
        issue: "Line 45: '- [ ] **MON-01**' is unchecked; lines 108-110 show MON-01 as Pending while MON-02/MON-03 are Complete"
    missing:
      - "Mark MON-01 as [x] in REQUIREMENTS.md"
human_verification:
  - test: "Confirm Aion freshness check routes correctly in production"
    expected: "For a live Aion symbol (e.g. BTCUSDT), yesterday_BTCUSDT key shows 'aion:fresh (N m ago)' detail instead of crossover_tid comparison"
    why_human: "Requires a running ClickHouse instance with live bar data to verify the freshness path end-to-end"
---

# Phase 18: Monitor Validation Gates — Verification Report

**Phase Goal:** Heartbeat, pre-write validation, and Stathera auditing work correctly for both Aion and Day-mode symbols without manual mode configuration
**Verified:** 2026-03-27T17:30:00Z
**Status:** gaps_found
**Re-verification:** No — initial verification

---

## Goal Achievement

### Observable Truths

| #   | Truth                                                                                                 | Status     | Evidence                                                                                                                                                                                     |
| --- | ----------------------------------------------------------------------------------------------------- | ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 1   | Aion symbol bars that cross midnight are NOT filtered by the pre-write gate                           | ✓ VERIFIED | `_filter_cross_midnight_pandas` builds `day_symbols` set from registry and only masks `df["symbol"].isin(day_symbols)` — Aion symbols never enter the set                                    |
| 2   | Day-mode symbol bars that cross midnight ARE still filtered by the pre-write gate                     | ✓ VERIFIED | Mask logic: `cross_mask & mask_day_mode` — day symbols in the set are filtered correctly; 6 prewrite tests all pass                                                                          |
| 3   | A DataFrame containing mixed Aion and Day symbols correctly filters only Day-mode cross-midnight bars | ✓ VERIFIED | `test_mixed_batch_only_day_filtered` in `tests/test_mon02_prewrite_gate.py` passes                                                                                                           |
| 4   | Stathera cross-midnight oracle excludes Aion bars from violation count                                | ✓ VERIFIED | `scripts/heartbeat/checks/stathera.py` line 280: `AND ouroboros_mode = 'day'` added to the cross-midnight query                                                                              |
| 5   | Aion symbols get a rolling-window freshness check instead of yesterday orphan completion              | ✓ VERIFIED | `yesterday.py` calls `_check_aion_freshness()` when `mode == "aion"`; 5 TestAionFreshness tests pass                                                                                         |
| 6   | Day-mode symbols still get the existing crossover_tid orphan completion check                         | ✗ FAILED   | 6 pre-existing TestYesterdayCompletion tests fail: BTCUSDT is aion in symbols.toml but tests don't mock get_symbol_entries(), so BTCUSDT routes to freshness check instead of day-mode check |
| 7   | Aion freshness window is configurable via environment variable                                        | ✓ VERIFIED | `scripts/heartbeat/config.py` lines 23-25: `AION_FRESHNESS_WINDOW_MIN = int(os.environ.get("OPENDEVIATIONBAR_AION_FRESHNESS_WINDOW_MIN", "30"))`                                             |
| 8   | When an Aion symbol has no bar within the freshness window, yesterday_completion reflects the issue   | ✓ VERIFIED | `_check_aion_freshness` sets `results[f"yesterday_{symbol}"] = False` when stale; aggregate `yesterday_completion` uses `is not False` gate                                                  |

**Score:** 7/8 truths verified (6 for automated checks, 1 truth partially fails due to test regression)

---

## Required Artifacts

| Artifact                                                | Expected                                           | Status               | Details                                                                                                                                                           |
| ------------------------------------------------------- | -------------------------------------------------- | -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `python/opendeviationbar/clickhouse/bulk_operations.py` | Per-symbol day-mode pre-write gate                 | ✓ VERIFIED           | `_get_day_symbols()` at line 157, `_filter_cross_midnight_pandas()` at line 173, `_filter_cross_midnight_polars()` at line 218; both use `is_in(day_symbols)`     |
| `scripts/heartbeat/checks/stathera.py`                  | Mode-aware cross-midnight oracle                   | ✓ VERIFIED           | `AND ouroboros_mode = 'day'` at line 280 in cross-midnight count query                                                                                            |
| `tests/test_mon02_prewrite_gate.py`                     | Pre-write gate unit tests for both modes           | ✓ VERIFIED           | 212 lines, 6 tests, all passing                                                                                                                                   |
| `scripts/heartbeat/checks/yesterday.py`                 | Mode-branching yesterday check with Aion freshness | ✓ VERIFIED           | `_check_aion_freshness()` and `_check_day_completion()` branches at lines 54-57                                                                                   |
| `scripts/heartbeat/config.py`                           | AION_FRESHNESS_WINDOW_MIN constant                 | ✓ VERIFIED           | Lines 23-25, env-configurable default 30                                                                                                                          |
| `tests/test_heartbeat_yesterday_junction.py`            | New Aion freshness test cases                      | ✓ VERIFIED (partial) | `TestAionFreshness` class with 5 tests all passing; however, 6 pre-existing `TestYesterdayCompletion` tests now fail due to missing mock for `get_symbol_entries` |

---

## Key Link Verification

| From                 | To                                 | Via                                                      | Status  | Details                                                                                                           |
| -------------------- | ---------------------------------- | -------------------------------------------------------- | ------- | ----------------------------------------------------------------------------------------------------------------- |
| `bulk_operations.py` | `opendeviationbar.symbol_registry` | `get_symbol_entries()` for per-symbol mode lookup        | ✓ WIRED | `from opendeviationbar.symbol_registry import get_symbol_entries` at line 163; called inside `_get_day_symbols()` |
| `bulk_operations.py` | filter logic                       | `is_in(day_symbols)` in pandas and Polars paths          | ✓ WIRED | `df["symbol"].isin(day_symbols)` at line 196; `pl.col("symbol").is_in(list(day_symbols))` at line 243             |
| `stathera.py`        | ClickHouse query                   | `ouroboros_mode = 'day'` filter in cross-midnight oracle | ✓ WIRED | Line 280: `AND ouroboros_mode = 'day'` confirmed present                                                          |
| `yesterday.py`       | `opendeviationbar.symbol_registry` | `get_symbol_entries()` for per-symbol mode lookup        | ✓ WIRED | `from opendeviationbar.symbol_registry import get_symbol_entries` at line 16                                      |
| `yesterday.py`       | ClickHouse                         | `max(close_time_us)` query for Aion freshness            | ✓ WIRED | `_check_aion_freshness` runs the query at lines 77-84                                                             |

---

## Data-Flow Trace (Level 4)

| Artifact                                | Data Variable     | Source                                              | Produces Real Data                 | Status    |
| --------------------------------------- | ----------------- | --------------------------------------------------- | ---------------------------------- | --------- |
| `yesterday.py` `_check_aion_freshness`  | `max_close_us`    | ClickHouse `max(close_time_us)` query               | Yes — real DB query on live data   | ✓ FLOWING |
| `yesterday.py` `_check_day_completion`  | `orphan_tid`      | ClickHouse `max(last_agg_trade_id)` WHERE yesterday | Yes — real DB query                | ✓ FLOWING |
| `bulk_operations.py` `_get_day_symbols` | `day_symbols` set | `get_symbol_entries()` registry read                | Yes — reads from symbols.toml SSoT | ✓ FLOWING |

---

## Behavioral Spot-Checks

| Behavior                    | Command                                                                                              | Result     | Status                                       |
| --------------------------- | ---------------------------------------------------------------------------------------------------- | ---------- | -------------------------------------------- |
| Pre-write gate tests        | `uv run --python 3.13 pytest tests/test_mon02_prewrite_gate.py -v`                                   | 6/6 passed | ✓ PASS                                       |
| Aion freshness tests        | `uv run --python 3.13 pytest tests/test_heartbeat_yesterday_junction.py::TestAionFreshness -v`       | 5/5 passed | ✓ PASS                                       |
| Pre-existing day-mode tests | `uv run --python 3.13 pytest tests/test_heartbeat_yesterday_junction.py::TestYesterdayCompletion -v` | 0/6 passed | ✗ FAIL                                       |
| Junction telemetry tests    | `uv run --python 3.13 pytest tests/test_heartbeat_yesterday_junction.py::TestJunctionTelemetry -v`   | 0/2 passed | ✗ FAIL (pre-existing, unrelated to Phase 18) |

**Note on junction test failures:** The `TestJunctionTelemetry` failures (`KeyError: 'junction_telemetry'`) were present before Phase 18 began. Confirmed by checking the pre-Phase-18 commit `eef81cb`. These are out-of-scope for Phase 18 verification.

---

## Requirements Coverage

| Requirement | Source Plan   | Description                                                                                           | Status      | Evidence                                                                                                                                                                                                                                     |
| ----------- | ------------- | ----------------------------------------------------------------------------------------------------- | ----------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| MON-01      | 18-02-PLAN.md | Yesterday completion check replaced with rolling-window freshness check for Aion symbols              | ✗ BLOCKED   | Implementation is complete and correct, but REQUIREMENTS.md line 45 still shows `[ ]` unchecked. Also, 6 TestYesterdayCompletion tests regressed because the new mode-branching code calls get_symbol_entries() but the tests don't mock it. |
| MON-02      | 18-01-PLAN.md | Day-mode pre-write gate in bulk_operations.py is conditional — skipped for Aion symbols               | ✓ SATISFIED | `_get_day_symbols()` + `_filter_cross_midnight_pandas/polars()` confirmed; all 6 prewrite tests pass                                                                                                                                         |
| MON-03      | 18-01-PLAN.md | Stathera audit query is mode-agnostic (trade-ID continuity checking works identically for both modes) | ✓ SATISFIED | Cross-midnight oracle has `AND ouroboros_mode = 'day'` to exclude Aion cross-midnight bars from false violation counts                                                                                                                       |

**MON-01 discrepancy:** The 18-02-SUMMARY.md declares `requirements-completed: [MON-01]` but REQUIREMENTS.md (the authority) still shows it as `[ ]` pending. The code is functionally correct, but the requirements file was not updated and 6 existing tests regressed.

---

## Anti-Patterns Found

| File                                         | Line    | Pattern                                                                                                                                                            | Severity   | Impact                                                                     |
| -------------------------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------- | -------------------------------------------------------------------------- |
| `tests/test_heartbeat_yesterday_junction.py` | 170-339 | `TestYesterdayCompletion` tests use BTCUSDT without mocking `get_symbol_entries()` — BTCUSDT is aion in symbols.toml so day-mode test assertions are never reached | 🛑 Blocker | 6 tests fail; breaks the test suite for Phase 18's core MON-01 requirement |

---

## Human Verification Required

### 1. Aion Freshness in Production

**Test:** On bigblack, run the heartbeat check and inspect the Telegram message or results dict for an Aion symbol.
**Expected:** `yesterday_BTCUSDT` shows `True` with detail `"aion:fresh (N m ago)"` where N is within the last 30 minutes.
**Why human:** Requires a running ClickHouse instance with live bar data from the sidecar.

---

## Gaps Summary

Phase 18 successfully delivered the core Aion-aware logic for all three requirements:

- MON-02: Per-symbol pre-write gate works correctly (6/6 tests pass)
- MON-03: Stathera cross-midnight oracle excludes Aion symbols (SQL filter confirmed)
- MON-01 implementation: Code is correct and the new Aion freshness tests pass (5/5)

However, two gaps prevent clean sign-off:

**Gap 1 (Blocker — test regression):** Phase 18 added `get_symbol_entries()` call inside `collect_yesterday_completion()`, which is correct behavior. But 6 pre-existing `TestYesterdayCompletion` tests were written assuming BTCUSDT would always use the day-mode path. Since BTCUSDT is `ouroboros_mode = "aion"` in `symbols.toml` (set by Phase 13), these tests now route to the freshness check and fail on day-mode assertions. The fix is to add `get_symbol_entries` mock to each `TestYesterdayCompletion` test, setting BTCUSDT to day-mode for those scenarios.

**Gap 2 (Minor — documentation):** `REQUIREMENTS.md` line 45 still shows `- [ ] **MON-01**` as unchecked. The Phase 18-02 SUMMARY claims completion, but the authoritative requirements tracker was not updated.

The two gaps share the same root concern: the day-mode crossover_tid path in `yesterday.py` is not fully verified with tests that correctly isolate it from real registry data.

---

_Verified: 2026-03-27T17:30:00Z_
_Verifier: Claude (gsd-verifier)_
