Eval Report: ci-pr-smoke

Profile: gdm-swebench-lite-v1 | Tasks: 2 | Pass rate: 100.0% | Cost: $0.0020

Task ID	Band	Score	Passed	Cost
python-security-fix-easy-001	easy	0.740	✓	$0.0010
typescript-security-fix-easy-001	easy	0.740	✓	$0.0010

Leaderboard Snapshot

Latest run: 9891df5b-b3e7-4904-ac6c-b756915e9d37 | Latest model: coder | Latest score: 0.740 | Recorded at: 2026-04-27T16:14:32.749084+00:00

Run ID	Model	Git SHA	Score	Created
9891df5b-b3e7-4904-ac6c-b756915e9d37	coder	4669773b4fbe9d507f1396f38777a1b36998faf3	0.740	2026-04-27T16:14:32.749084+00:00
92541717-ec8b-4d17-a4b5-4e0db7bc9857	coder	4669773b4fbe9d507f1396f38777a1b36998faf3	0.740	2026-04-27T16:14:32.680215+00:00
dee1db7c-ce07-4710-b07a-833848794aa8	coder	4669773b4fbe9d507f1396f38777a1b36998faf3	0.740	2026-04-27T16:14:32.387794+00:00
491336af-ea5d-42d4-821e-ea1552be6574	coder	4669773b4fbe9d507f1396f38777a1b36998faf3	0.740	2026-04-27T16:14:32.314045+00:00
a8caf138-e4e4-4f26-a352-26abb1321769	coder	4669773b4fbe9d507f1396f38777a1b36998faf3	0.740	2026-04-27T16:14:32.249922+00:00

Taxonomy	Failures	Bar
wrong-logic	158	##############################################################################################################################################################

Run ID	Task ID	Taxonomy	Score	Cost	Created
9891df5b-b3e7-4904-ac6c-b756915e9d37	typescript-security-fix-easy-001	wrong-logic	0.740	$0.0010	2026-04-27T16:14:32.749084+00:00
92541717-ec8b-4d17-a4b5-4e0db7bc9857	python-security-fix-easy-001	wrong-logic	0.740	$0.0010	2026-04-27T16:14:32.680215+00:00
dee1db7c-ce07-4710-b07a-833848794aa8	python-recovery-easy-001	wrong-logic	0.740	$0.0010	2026-04-27T16:14:32.387794+00:00
491336af-ea5d-42d4-821e-ea1552be6574	typescript-config-easy-001	wrong-logic	0.740	$0.0010	2026-04-27T16:14:32.314045+00:00
a8caf138-e4e4-4f26-a352-26abb1321769	python-config-easy-001	wrong-logic	0.740	$0.0010	2026-04-27T16:14:32.249922+00:00
ccf3f268-459e-4d9c-9d19-b73407b37a0c	typescript-refactor-easy-001	wrong-logic	0.740	$0.0010	2026-04-27T16:14:32.194352+00:00
c7faff11-c7a3-4fc6-9936-2ff18b547349	python-refactor-easy-001	wrong-logic	0.740	$0.0010	2026-04-27T16:14:32.151347+00:00
13b72298-2ff9-44c0-948c-9976e3c3ed71	typescript-multi-file-easy-001	wrong-logic	0.740	$0.0010	2026-04-27T16:14:32.101694+00:00
5e9627a1-0cda-4e92-afb3-745e8fc0fec3	python-multi-file-easy-001	wrong-logic	0.740	$0.0010	2026-04-27T16:14:32.044298+00:00
debd5260-6599-48a8-9713-fb0cb60972d4	typescript-test-writing-easy-001	wrong-logic	0.740	$0.0010	2026-04-27T16:14:31.977927+00:00

pass_rate vs cost_usd (Pareto frontier marked with *)
* [####################] 100.0% @ $0.0055  (coder)