### test (26 tasks) - separated: none (every rung's interval overlaps another's)

| rung | accepted | share | 95% interval | answer correct | no answer | median wall s | median model s | cost USD |
| -- | -- | -- | -- | -- | -- | -- | -- | -- |
| local-qwen3.8-27b | 10/26 | 38% | 22%-57% | 11 | 0 | 30.5 | 13.6 | 0.0000 (26 receipts) |
| glm-5.3-flash | 8/26 | 31% | 17%-50% | 8 | 17 | 175.1 | n/a | not measured |
| glm-5.3 | 16/26 | 62% | 43%-78% | 16 | 5 | 74.9 | n/a | not measured |
| openrouter-nemotron-ultra | 8/26 | 31% | 17%-50% | 12 | 0 | 122.5 | 105.9 | 0.0000 (26 receipts) |
| openrouter-nemotron-super | 2/26 | 8% | 2%-24% | 13 | 2 | 190.0 | 153.6 | 0.0000 (24 receipts) |

### code_generation (26 tasks) - separated: none (every rung's interval overlaps another's)

| rung | accepted | share | 95% interval | answer correct | no answer | median wall s | median model s | cost USD |
| -- | -- | -- | -- | -- | -- | -- | -- | -- |
| local-qwen3.8-27b | 21/26 | 81% | 62%-91% | 21 | 0 | 21.7 | 4.8 | 0.0000 (26 receipts) |
| glm-5.3-flash | 18/26 | 69% | 50%-83% | 18 | 6 | 51.6 | n/a | not measured |
| glm-5.3 | 24/26 | 92% | 76%-98% | 24 | 0 | 24.9 | n/a | not measured |
| openrouter-nemotron-ultra | 20/26 | 77% | 58%-89% | 21 | 0 | 69.9 | 24.9 | 0.0000 (26 receipts) |
| openrouter-nemotron-super | 17/26 | 65% | 46%-81% | 17 | 0 | 53.3 | 28.0 | 0.0000 (26 receipts) |
