EVALARC / DEVELOPMENT AUDIT

Test the grader.
Then trust the signal.

Behavioral controls for task-specific agent evaluations. Correct and faulty submissions are evaluated against the same externally enforced contract.

8/8
declared faults detected
1.000
reference correctness
docker
execution backend

Does the grader detect plausible defects?

Negative controlTargetCandidate scoreResultEvidence
ack-without-workbasic0.000Detectedjson-types-and-overwrite, read-write-delete, seeded-state-machine
memory-onlypersistence0.783Detectedrestart-preserves-acknowledged-write, restart-preserves-cas, restart-preserves-transaction
partial-batchtransactions0.900Detectedrollback-batch
cas-always-winscompare_swap0.850Detectedcas-match-and-mismatch, cas-type-sensitivity
boolean-equals-onecompare_swap0.925Detectedcas-type-sensitivity
delete-noopbasic0.650Detectedread-write-delete, seeded-state-machine
accept-nonstring-keysvalidation0.925Detectedreject-and-continue
commit-on-exitcrash_recovery0.900Detectedsigkill-after-acknowledgement, sigkill-after-atomic-batch, sigkill-after-delete

Positive control

basic1.003/3
validation1.002/2
transactions1.002/2
compare_swap1.002/2
persistence1.003/3
crash_recovery1.003/3

Reproduction record

Task: durable-kv v0.1.0
Seeds: [17]
Grader SHA-256: 2283f5c82c24ac536c4467f5833333fc7098adb46839238fdc56eca017ef2039
Cases SHA-256: 842a16a1e65700cde92ee5f5aaaa10d494a9fc9d71bd2d99fb68147b19aad63a
Image ID: sha256:78387bc3881b8273120a12ebe6c1ab22b018ccc2c9adf565ae1ac9b536e184ea
Created: 2026-09-14T03:11:33.142666+00:00

Complete JSON evidence for all controls