kshana-bench leaderboard

Result files scored under kshana-bench/2.0.0. A row says this file scored this number on this split. It does not mean any product has been rated or approved by anyone, and no organisation is named as an endorser.

Advisory: modelled or measured against stated references, not an approval, certification or statement of conformance. A score describes a log or corpus, never a product.

Task (a): detection on receiver logs

Split: dev
System	AUC	P_d at P_fa cap (upper bound)	Provenance	Truth SHA-256
kshana receiver-trust agc_k_sigma (0.35 default monitor statistic) kshana-bench/2.0.0	0.981	0.875	scored	`334946332bb0`
kshana receiver-trust cn0_drop_db (0.35 default monitor statistic) kshana-bench/2.0.0	0.952	0.904	scored	`334946332bb0`
kshana receiver-trust jam_ind_threshold (0.35 default monitor statistic) kshana-bench/2.0.0	0.909	0.147	scored	`334946332bb0`

Split: holdout
System	AUC	P_d at P_fa cap (upper bound)	Provenance	Truth SHA-256
kshana receiver-trust agc_k_sigma (0.35 default monitor statistic) kshana-bench/2.0.0	1.000	0.000	scored	`38b7d5e8b158`
kshana receiver-trust jam_ind_threshold (0.35 default monitor statistic) kshana-bench/2.0.0	0.610	0.148	scored	`38b7d5e8b158`
kshana receiver-trust cn0_drop_db (0.35 default monitor statistic) kshana-bench/2.0.0	0.564	0.562	scored	`38b7d5e8b158`

Split: test
System	AUC	P_d at P_fa cap (upper bound)	Provenance	Truth SHA-256
kshana receiver-trust cn0_drop_db (0.35 default monitor statistic) kshana-bench/2.0.0	0.864	0.522	scored	`ee721e0bf7dc`
kshana receiver-trust agc_k_sigma (0.35 default monitor statistic) kshana-bench/2.0.0	0.766	0.000	scored	`ee721e0bf7dc`
kshana receiver-trust jam_ind_threshold (0.35 default monitor statistic) kshana-bench/2.0.0	0.556	0.104	scored	`ee721e0bf7dc`

Task (b): detection on IQ

Split: test
System	AUC	P_d at P_fa cap (upper bound)	Provenance	Truth SHA-256
kshana iq monitors: cn0_drop (CUSUM) kshana-bench/2.0.0	0.785	n/a	carried: docs/design/DETECTION-ROC.md section 3: the R1 campaign of tests/iq_campaign_roc.rs, run 2026-10-13; not re-run for bench v1. Bar R1 (AUC >= 0.95) was NOT MET.	`000000000000`

Task (c): timing holdover prediction

Split: dev
System	RMSE 3600 s (ns)	RMSE 300 s (ns)	RMSE 7200 s (ns)	Provenance	Truth SHA-256
kshana-bench baseline: linear kshana-bench/2.0.0	0.128	0.019	0.198	scored	`2b71fa325bff`
kshana-bench baseline: quadratic kshana-bench/2.0.0	0.135	0.019	0.235	scored	`2b71fa325bff`
kshana-bench baseline: hold kshana-bench/2.0.0	26.876	2.238	53.778	scored	`2b71fa325bff`

Split: test
System	RMSE 3600 s (ns)	RMSE 300 s (ns)	RMSE 7200 s (ns)	Provenance	Truth SHA-256
kshana-bench baseline: linear kshana-bench/2.0.0	0.162	0.023	0.243	scored	`b2ed58864e98`
kshana-bench baseline: quadratic kshana-bench/2.0.0	0.183	0.024	0.290	scored	`b2ed58864e98`
kshana-bench baseline: hold kshana-bench/2.0.0	24.612	2.050	49.314	scored	`b2ed58864e98`

Task (d): out-of-distribution optimism gap

Split: dev
System	AUC in	AUC out	Optimism gap	Flag	Provenance	Truth SHA-256
impairment-eval baseline: fused(max-z) kshana-bench/2.0.0	0.998	0.982	0.015	within tolerance	scored	`f69917c70201`
impairment-eval baseline: sqm-imbalance kshana-bench/2.0.0	0.867	0.866	0.001	within tolerance	scored	`f69917c70201`
impairment-eval baseline: agc-excess kshana-bench/2.0.0	0.854	0.828	0.026	within tolerance	scored	`f69917c70201`
impairment-eval baseline: raim-parity kshana-bench/2.0.0	0.655	0.642	0.013	within tolerance	scored	`f69917c70201`
impairment-eval baseline: energy(cn0-drop) kshana-bench/2.0.0	0.610	0.572	0.038	within tolerance	scored	`f69917c70201`

Split: test
System	AUC in	AUC out	Optimism gap	Flag	Provenance	Truth SHA-256
impairment-eval baseline: fused(max-z) kshana-bench/2.0.0	0.998	0.983	0.015	within tolerance	scored	`1dddb8f6f0d5`
impairment-eval baseline: sqm-imbalance kshana-bench/2.0.0	0.882	0.882	0.001	within tolerance	scored	`1dddb8f6f0d5`
impairment-eval baseline: agc-excess kshana-bench/2.0.0	0.848	0.819	0.030	within tolerance	scored	`1dddb8f6f0d5`
impairment-eval baseline: raim-parity kshana-bench/2.0.0	0.682	0.672	0.010	within tolerance	scored	`1dddb8f6f0d5`
impairment-eval baseline: energy(cn0-drop) kshana-bench/2.0.0	0.595	0.565	0.030	within tolerance	scored	`1dddb8f6f0d5`

Every score is modelled unless a row says an external oracle agreed. Licence of the specification and manifests: CC-BY-4.0 / CC0 proposed, owner decision pending. The real-data holdout is not secret: see the specification, section 4.