Jev bench results (SAMPLE: dry-run stub data)

Model: not recorded. Thinking: not recorded.

Jev connection: 2 of 4 with-Jev runs that reached the model got a Jev answer (50%). Automatic arm: 0 of 2 model-reached B runs called Jev; 2 did not call Jev.

Jev round trip: n=2, median 36.4 ms, p90 43.0 ms, p95 43.8 ms.

Tokens: not measured.

Agent model spend (Pi usage cost, summed): A $0.0000; B $0.0000; C $0.0000.

History: the 2026-09-22 two-arm Pi run recorded 150 pairs; after 43 with-Jev runs reached the local model, 127.0.0.1:8000 stopped accepting connections (Pi reported Connection error with zero tokens, on both arms) and the remaining 107 runs were server failures, not model speed.

Stop: all 2 triplets recorded

A directB automaticC forced
Accuracy: share of items correct
0.0%25.0%50.0%75.0%100.0%All itemsAll items · A direct: 100.0%100.0%All items · B automatic: 100.0%100.0%All items · C forced: 100.0%100.0%jev_screenjev_screen · A direct: 100.0%100.0%jev_screen · B automatic: 100.0%100.0%jev_screen · C forced: 100.0%100.0%jev_verifyjev_verify · A direct: 100.0%100.0%jev_verify · B automatic: 100.0%100.0%jev_verify · C forced: 100.0%100.0%unit: share correct
Evaluator overhead: items per minute (items / summed wall time)
0.00/min472.41/min944.81/min1417.22/min1889.63/minAll itemsAll items · A direct: 1617.85/min1617.85/minAll items · B automatic: 89.51/min89.51/minAll items · C forced: 88.72/min88.72/minjev_screenjev_screen · A direct: 1889.63/min1889.63/minjev_screen · B automatic: 89.36/min89.36/minjev_screen · C forced: 99.56/min99.56/minjev_verifyjev_verify · A direct: 1414.42/min1414.42/minjev_verify · B automatic: 89.66/min89.66/minjev_verify · C forced: 80.02/min80.02/minunit: items/min
Time consumed: total wall time
0.00 s0.34 s0.68 s1.01 s1.35 sAll itemsAll items · A direct: 0.07 s0.07 sAll items · B automatic: 1.34 s1.34 sAll items · C forced: 1.35 s1.35 sjev_screenjev_screen · A direct: 0.03 s0.03 sjev_screen · B automatic: 0.67 s0.67 sjev_screen · C forced: 0.60 s0.60 sjev_verifyjev_verify · A direct: 0.04 s0.04 sjev_verify · B automatic: 0.67 s0.67 sjev_verify · C forced: 0.75 s0.75 sunit: seconds

Headline (complete triplets)

triplets2
wall_A_median_s0.03709
wall_A_p90_s0.04135
wall_A_p95_s0.04189
jev_usd_A0
called_jev_A0
called_jev_share_A0
wall_B_median_s0.6703
wall_B_p90_s0.6712
wall_B_p95_s0.6713
jev_usd_B0
called_jev_B0
called_jev_share_B0
wall_C_median_s0.6763
wall_C_p90_s0.7351
wall_C_p95_s0.7425
jev_usd_C1.008e-05
called_jev_C2
called_jev_share_C1
wall_b_minus_a_median_s0.6333
wall_b_minus_a_p90_s0.6384
wall_b_minus_a_p95_s0.6391
wall_b_minus_a_sign_p0.5
wall_c_minus_a_median_s0.6392
wall_c_minus_a_p90_s0.6938
wall_c_minus_a_p95_s0.7006
wall_c_minus_a_sign_p0.5

Runs

runtoolstatususe gateanswercorrectwall sJev calls
dry-screen-01.Ajev_screenokn/a (arm has no Jev)cleanTrue0.030
dry-screen-01.Bjev_screenokdid not call JevcleanTrue0.670
dry-screen-01.Cjev_screenokused JevcleanTrue0.601
dry-verify-01.Ajev_verifyokn/a (arm has no Jev)verifiedTrue0.040
dry-verify-01.Bjev_verifyokdid not call JevverifiedTrue0.670
dry-verify-01.Cjev_verifyokused JevverifiedTrue0.751