opendeviationbar-py · bounded campaign · started 2026-07-22
Before a promoted feature can be called real or useful, the instrument that judges it must itself be proven on data where the right answer is known. This bounded loop grounds the 20 candidate instruments (#0–#19) — one per iteration, in dependency order — then STOPS. Every iteration emits one HTML page, appended to the ledger at the bottom of this index.
Each candidate instrument sits an entrance exam: the §4 five-control battery (positive / negative-null / substitution / power-calibration / invariance-leakage), plus its instrument-specific Harden item and a published detection envelope. It ends with a terminal verdict:
DISCOVERED → UNDER EVALUATION → GROUNDED (or REJECTED / UNVERIFIABLE-PARK)
Only a GROUNDED instrument may later judge a real feature. The graduation tally is the loop's STOP condition: when all 20 carry a terminal verdict, the loop publishes a completion page and ends.
readonly=2, SELECT only, both substrates.systemd-run --user --scope cap on bigblack. Cap-kills are evidence; the cap is never raised./loop orchestrates on the laptop and dispatches every read + compute step to bigblack over SSH. Published to /home/nasimubd/sites.| Phase | Instruments (in order) |
|---|---|
| SEAL | real-data-only control substrate self-validation (first firing) |
| Substrate | #1 future-perturbation invariance → #0 CPCV purge/embargo (both GROUNDED or HALT) |
| Realness | #2 block-perm null → #3 effective-n+FDR → #6 SFI → #8 DSR → #9 PBO → #10 Harvey-Liu-Zhu |
| Usefulness | #4 Rank IC → #5 IC-decay → #7 MDA → #19 quantile monotonicity |
| Conditional | #11 CMI → #12 knockoffs → #13 DML |
| Increment / mechanism | #14 spanning → #15 mechanism-intensity |
| Robustness / detector | #16 structural-break → #17 plateau → #18 detector lead-lag |
Full per-instrument Admit / Graduate / Harden rows: METRIC-EVALUATION-FRAMEWORK.md §7. Metric tables twin: The Realness Question.
supersedes:). The conditional axis stays covered by #11 (GROUNDED) + #13.
One row per firing. The loop appends below the marker; rows are never edited or deleted (corrections are new rows with supersedes:). Each row links the iteration's own HTML page.
| # | Date | Instrument / step | Verdict | Page |
|---|---|---|---|---|
| 1 | 2026-07-22 | SEAL · real-data-only control-substrate self-validation | SEAL-PASS lab open | iter 1 → |
| 2 | 2026-07-22 | #1 · future-perturbation invariance (causality floor) | GROUNDED | iter 2 → |
| 3 | 2026-07-22 | #0 · CPCV purge/embargo (leakage-free partition) | CHECKPOINT 6/7 · PBO open | iter 3 → |
| 4 | 2026-07-22 | #0 · CPCV purge/embargo (supersedes iter 3) — ★ substrate gate PASS | GROUNDED | iter 4 → |
| 5 | 2026-07-22 | #2 · block-permutation shuffled-label null (realness) · P2 | GROUNDED | iter 5 → |
| 6 | 2026-07-22 | #3 · effective-n deflation + BH-FDR (realness) · P2 | GROUNDED | iter 6 → |
| 7 | 2026-07-23 | #6 · SFI single-feature OOS (usefulness) · P2 | GROUNDED | iter 7 → |
| 8 | 2026-07-23 | #8 · Deflated Sharpe Ratio (on SFI paths) · P2 | CHECKPOINT 3/5 · SR0 calibration | iter 8 → |
| 9 | 2026-07-23 | #8 · Deflated Sharpe Ratio (supersedes iter 8) — permutation SR0 | GROUNDED | iter 9 → |
| 10 | 2026-07-23 | #9 · PBO / CSCV (on the SFI trial matrix) · P2 | CHECKPOINT 1/7 · null≈.5 ✓, thresholds marginal | iter 10 → |
| 11 | 2026-07-23 | #9 · PBO / CSCV (supersedes iter 10) — within-block IC + full-shuffle null + K=100 | GROUNDED 7/7 · 6 seeds | iter 11 → |
| 12 | 2026-07-23 | #10 · Harvey–Liu–Zhu t≥3 (M_eff) · P2 · realness axis complete | GROUNDED 6/6 · 5 seeds · perm-calibrated FWER | iter 12 → |
| 13 | 2026-07-23 | #4 · Rank IC + ICIR + HAC-t · usefulness axis opens | CHECKPOINT 2/5 · injection fixed · period-IC rewrite | iter 13 → |
| 14 | 2026-07-23 | #4 · Rank IC (supersedes iter 13) — period-IC rewrite | CHECKPOINT 4/5 robust · FPR lag-tension → operator | iter 14 → |
| 15 | 2026-07-23 | #4 · Rank IC (supersedes iter 13, 14) — #10 perm-FPR resolves the flag | GROUNDED 5/5 · 5 seeds · first usefulness | iter 15 → |
| 16 | 2026-07-23 | #5 · IC-decay / half-life · usefulness | GROUNDED 4/4 · 5 seeds · overlap→HAC load-bearing | iter 16 → |
| 17 | 2026-07-23 | #7 · MDA + clustered-MDA (ONC) · usefulness | GROUNDED 5/5 · 5 seeds · ONC defeats substitution | iter 17 → |
| 18 | 2026-07-23 | #19 · Quantile monotonicity + PT/RW MR · usefulness | CHECKPOINT 5/6 base · per-test perm FPR · 3 open | iter 18 → |
| 19 | 2026-07-23 | #19 · Quantile monotonicity (supersedes iter 18) — tie-robust + both-halves RW | GROUNDED 6/6 · 5 seeds · usefulness roster complete | iter 19 → |
| 20 | 2026-07-23 | #11 · Conditional MI I(f;Y\|S) · conditional axis opens | CHECKPOINT 5/5 slice · 2 seeds · marg≈0 · iid/AR(1)+high-dim-S deferred | iter 20 → |
| 21 | 2026-07-23 | #11 · Conditional MI (supersedes iter 20) — all 4 nulls + high-dim-S + imperfect-S envelope | GROUNDED 8/8 · 2 seeds · AR(1) no-inflate · conditional axis | iter 21 → |
| 22 | 2026-07-23 | #12 · Model-X Knockoffs · conditional axis · slice 1 | CHECKPOINT MVR power 0.95 · iid-FDR inflates 0.25 → block/TSKI next | iter 22 → |
| 23 | 2026-07-23 | #12 · Model-X Knockoffs · slice 2 — course-correction | CHECKPOINT FDR≠autocorr (falsified); plain-knockoff anti-conservative, knockoff+ 0.0; interaction RF-W 1.0 vs lasso 0.0 | iter 23 → |
| 24 | 2026-07-24 | #12 · Model-X Knockoffs · slice 3 — power contour | CHECKPOINT wide panel + KnockoffFilter; power .71→1.0 (IC .03→.07), FDP ~1.5× q → conservative-q (PATTERN #2) | iter 24 → |
| 25 | 2026-07-24 | #12 · Model-X Knockoffs · slice 4 — standard construction exhausted | OPERATOR RULING power ceiling 0.67 @IC=.03; conservative-q FAILS (FDR stuck ~1.3×); recommend route→#13 | iter 25 → |