DashboardProbesRealness Loop › Iter 7 · #6

iteration 7 · 2026-07-23 · P2 · laptop-drives-bigblack

🎯 #6 SFI single-feature OOS GROUNDED

Train on one feature alone over the leakage-free CPCV substrate and ask: does it beat a coin toss out-of-sample? This is the per-feature return object that later unlocks the Deflated Sharpe and PBO tests. It grounds on 869,833 real bars.

z* = 2.13
threshold from the null (not 1.0)
31.5%
FPR if you naively use z=1.0
0.2 → 0.1
episodic κ_min (robust rescue)
5 / 20
instruments grounded
Preflight (resource-only): load1 0.49 · 42 GiB available · si/so ~0 · ClickHouse active, readonly=2. Compute wall ~11 s. Runs on the grounded #0 CPCV substrate (50 disjoint leave-one-group-out folds, purge+embargo).
Branch note: cycle-1 PR #657 was squash-merged (through #2); #3 was carried forward and this iteration opens cycle-2 PR #658 (carries #3 + #6 onward).

In plain English

To judge whether a single feature is useful on its own, we let it make out-of-sample bets across 50 time-slices and score how consistently it wins. The score's pass-mark is not a round number like 1 — we learn it from shuffled (no-signal) data: here the real bar is 2.13. If you'd naively used 1.0, you'd wrongly call a useless feature "useful" 31.5% of the time. And a feature whose edge lives in only a few regimes is invisible to the standard averaging score — so we add a concentration-robust readout that catches it, halving the concentration a feature needs before we can see it.

Gate status — all pass

GateResult
z* from null + FPRz* = 2.13 (null 95th pct, >1.0); FPR(z*) 0.028 ≤ α; FPR at naive z=1.0 = 0.315PASS
Power (known-positive)real look-ahead: .75 → .85 → .85 → .93; N_min 20,000PASS
Known-duplicateexact clone → identical z_SFIPASS
Harden (episodic)z=mean/sd κ_min 0.2; concentration-robust κ_min 0.1 (rescues κ=0.1)PASS

Why z* must come from the null

The CPCV fold scores are correlated, so the null distribution of the z-score is much wider than the textbook N(0,1). A naive pass-mark of z=1.0 therefore lets through 31.5% of pure-noise features. Deriving z* from the block-permuted null (2.13) restores false-positive control to 2.8% — exactly the §7 "PREREG from null, not 1.0" requirement.

The load-bearing Harden — concentration robustness

A feature whose edge is concentrated in a fraction κ of contiguous regimes produces a few strong folds and many flat folds → high fold-variance → the dispersion-penalizing z = mean/sd is suppressed and silently kills it (κ_min 0.2). The concentration-robust statistic (folds beating the null per-fold 95th percentile) catches the concentrated signal → κ_min drops to 0.1, rescuing the κ=0.1 case mean/sd misses. Regime-concentrated features are routed to this robust readout / #18 detector, never boneyard-ed.

Fold geometry (verify-before-report)

First run used C(10,2)=45 overlapping folds → correlated scores inflated z* to 8.4 (≫ the ~3.1 reference) and power stalled at 0.72. Switched to 50 disjoint leave-one-group-out folds (§7's F=50) → low correlation → z* = 2.13 → power recovers to 0.93. A fold-design correction, not a threshold tune.

Verdict & what's next

GROUNDED — the per-feature return object is validated.

Next iteration → #8 Deflated Sharpe Ratio (on SFI paths) — rides on this return object to deflate for the number of features tried.