DashboardProbesRealness Loop › Iter 3 · #0

iteration 3 · 2026-07-22 · laptop-drives-bigblack

🧮 #0 CPCV purge/embargo CHECKPOINT · in-progress

The leakage-free scoreboard every out-of-sample number rides on. On 870,304 real bars, 6 of 7 substrate gates pass — including the crucial proof that purge/embargo is load-bearing. One gate (the PBO-null calibration) has an open residual, so #0 does not graduate yet. The substrate gate stays pending.

6 / 7
substrate gates pass
t = 169
naive-kfold leak (→ 0 after purge)
0.55%
signal-recovery bias (purge preserves)
PBO 0.595
open · vs band [.45,.55]
Preflight (resource-only): load1 0.19 · 40 GiB free · si/so 0 · ClickHouse active, readonly=2. Compute wall ~34 s, peak RSS 521 MB (caps 5c/5G/no-swap, never hit). Empirical: 34.85% of consecutive bars close within 3s → ample real label-overlap leakage surface.

Why a checkpoint (not a verdict)

#0 is the substrate gate — everything downstream rides on it, so it does not graduate on a partial or hand-waved pass. Six gates are solid; the seventh (PBO-null ∈ [0.45,0.55]) sits at 0.595. Rather than p-hack the frozen criterion or drop it, #0 is checkpointed and resumes next firing with a pre-registered null. Default = not-grounded.

Gate status

GateResult
Fold generator0 label-window overlaps survive purge+embargo (naive has many)PASS
Leakage gatenaive random k-fold straddles 252,512 overlapping pairs, leak t = 169; CPCV+purge leaves 0 straddlingPASS
Positive recoverylook-ahead IC* recovered by CPCV OOS, 0.55% bias, power 1.0PASS
N_min(H) surfaceevents/fold 290,101; N_min lag5 ρ0.14→30k, lag20 ρ0.079→100k; dialed-IC → PARKPASS
Harden (zoo→substrate)KP recovered · KN silent (0/15 folds) · KD identicalPASS
Null FPRfamily-wise OOS-IC FPR 0.000 ≤ 0.05PASS
Null PBO ∈ [.45,.55]0.595 (S=10, 252 splits)OPEN

Headline: purge/embargo is load-bearing

Because 34.85% of consecutive bars close within the 3s label window, a naïve random k-fold splits 252,512 overlapping-label pairs across train/test — and their leaked label correlation reads a t-stat of 169 (a torrent of spurious skill). The CPCV contiguous-group + purge + embargo partition removes every straddling overlap (0 pairs, 0 contamination) while preserving genuine signal (0.55% recovery bias). The leakage-free scoreboard works.

The open item — PBO-null (honest account)

The block-permuted null gave PBO = 0.595, just above [0.45,0.55]. Diagnosis (verify-before-report): first run PBO=0.90 (null trials reused 2 features → broke CSCV exchangeability; fixed → 0.557); raising resolution to 252 splits moved it to 0.595 — a small systematic residual, not noise: a skill-less fixed rule has ~0 global IC, weakly anti-correlating its IS/OOS performance (winner's-curse), nudging PBO above 0.5. This is a property of the PBO/CSCV estimator (instrument #9's tool), not the #0 partition. Next firing resolves it with a pre-registered independent-IS/OOS null, then #0 reaches its terminal verdict.

Bugs caught this iteration

  1. block_perm length bug — iterated permuted block indices as start positions → a garbage 870,400-length array (vs n=870,304), silently corrupting PBO before erroring. Fixed to map through the real block starts.
  2. Leakage gate under-powered — the first metric looked only at the nearest train bar per test bar at contiguous boundaries (<8 pairs/combo → M0=0). Reformulated to the random-k-fold straddling-overlap correlation (252k pairs, t=169).