Eval Report: ci-pr-smoke

Profile: gdm-swebench-lite-v1 | Tasks: 2 | Pass rate: 100.0% | Cost: $0.0020

Task IDBandScorePassedCost
python-security-fix-easy-001easy0.740$0.0010
typescript-security-fix-easy-001easy0.740$0.0010

Leaderboard Snapshot

Latest run: 2bd9018c-9c51-4e57-852b-d51c5b1587f4 | Latest model: coder | Latest score: 0.740 | Recorded at: 2026-04-27T16:26:08.507750+00:00

Recent Trend

Run IDModelGit SHAScoreCreated
2bd9018c-9c51-4e57-852b-d51c5b1587f4coder4669773b4fbe9d507f1396f38777a1b36998faf30.7402026-04-27T16:26:08.507750+00:00
419a0c12-e37d-4811-ad3e-6a6efc77ba3ecoder4669773b4fbe9d507f1396f38777a1b36998faf30.7402026-04-27T16:26:08.419359+00:00
ab88deeb-da0b-433c-a317-1aefd4c4f701coder4669773b4fbe9d507f1396f38777a1b36998faf30.7402026-04-27T16:26:07.936876+00:00
97f9b1d1-902b-4732-96f8-bf3974780affcoder4669773b4fbe9d507f1396f38777a1b36998faf30.7402026-04-27T16:26:07.846515+00:00
11e49a8c-d127-4573-8326-aceb197f0f2dcoder4669773b4fbe9d507f1396f38777a1b36998faf30.7402026-04-27T16:26:07.787647+00:00

Failure Breakdown

TaxonomyFailuresBar
wrong-logic252
############################################################################################################################################################################################################################################################

Recent Failures

Run IDTask IDTaxonomyScoreCostCreated
2bd9018c-9c51-4e57-852b-d51c5b1587f4typescript-security-fix-easy-001wrong-logic0.740$0.00102026-04-27T16:26:08.507750+00:00
419a0c12-e37d-4811-ad3e-6a6efc77ba3epython-security-fix-easy-001wrong-logic0.740$0.00102026-04-27T16:26:08.419359+00:00
ab88deeb-da0b-433c-a317-1aefd4c4f701python-recovery-easy-001wrong-logic0.740$0.00102026-04-27T16:26:07.936876+00:00
97f9b1d1-902b-4732-96f8-bf3974780afftypescript-config-easy-001wrong-logic0.740$0.00102026-04-27T16:26:07.846515+00:00
11e49a8c-d127-4573-8326-aceb197f0f2dpython-config-easy-001wrong-logic0.740$0.00102026-04-27T16:26:07.787647+00:00
41023bc0-52c8-4f36-824f-734ff692b8a2typescript-refactor-easy-001wrong-logic0.740$0.00102026-04-27T16:26:07.702442+00:00
510e34d9-e187-42cf-b3d5-bfc791ab2d85python-refactor-easy-001wrong-logic0.740$0.00102026-04-27T16:26:07.616440+00:00
3fffafb4-8344-4f1a-a8d7-6ec071e415actypescript-multi-file-easy-001wrong-logic0.740$0.00102026-04-27T16:26:07.492856+00:00
4b79bedb-97b2-4507-9df2-ea98ced97327python-multi-file-easy-001wrong-logic0.740$0.00102026-04-27T16:26:07.391639+00:00
d00c3723-b217-4246-92db-079d056daf51typescript-test-writing-easy-001wrong-logic0.740$0.00102026-04-27T16:26:07.304229+00:00

Cost Frontier

pass_rate vs cost_usd (Pareto frontier marked with *)
* [####################] 100.0% @ $0.0054  (coder)