200 decisions · accuracy 95.5% · ECE 0.0389
cost $0.0038 · p50 0.741s · p99 3.202s
judge `jev` · model `typesafe-ai/jev` · backend `gateway` · recomputed 2026-09-19T09:09:12+00:00 (original run time not recorded) · judge-audit 0.2.0
Zero observed errors through the most confident 73.0%
(146 decisions, confidence ≥ 0.92).
Retrospective on this dataset — not a production guarantee.
| coverage | accuracy | min confidence | n |
|---|---|---|---|
| 5% | 100.0% | 1.00 | 10 |
| 15% | 100.0% | 1.00 | 30 |
| 25% | 100.0% | 1.00 | 50 |
| 35% | 100.0% | 1.00 | 70 |
| 45% | 100.0% | 1.00 | 90 |
| 55% | 100.0% | 0.99 | 110 |
| 65% | 100.0% | 0.96 | 130 |
| 75% | 99.3% | 0.90 | 150 |
| 85% | 99.4% | 0.78 | 170 |
| 95% | 97.9% | 0.56 | 190 |
| bin | avg confidence | accuracy | n |
|---|---|---|---|
| 0.5-0.6 | 0.537 | 66.7% | 15 |
| 0.6-0.7 | 0.644 | 62.5% | 8 |
| 0.7-0.8 | 0.750 | 100.0% | 10 |
| 0.8-0.9 | 0.853 | 100.0% | 16 |
| 0.9-1.0 | 0.988 | 99.3% | 151 |
A perfectly honest judge sits on the diagonal: avg confidence == accuracy in every bin.