Jev bench results

Model: opencode-go/deepseek-v4.1-flash. Thinking: high.

Jev connection: 149 of 300 with-Jev runs that reached the model got a Jev answer (50%). Automatic arm: 0 of 150 model-reached B runs called Jev; 150 did not call Jev.

Jev round trip: n=157, median 464.6 ms, p90 1245.3 ms, p95 1468.8 ms.

Tokens (Pi usage fields, summed): A input 447114 / output 42804 / cache read 30592 / cache write 0; B input 438467 / output 49315 / cache read 39680 / cache write 0; C input 1230791 / output 167029 / cache read 1860608 / cache write 0.

Agent model spend (Pi usage cost, summed): A $0.0928; B $0.0955; C $0.2904.

History: the 2026-09-22 two-arm Pi run recorded 150 pairs; after 43 with-Jev runs reached the local model, 127.0.0.1:8000 stopped accepting connections (Pi reported Connection error with zero tokens, on both arms) and the remaining 107 runs were server failures, not model speed.

Stop: all 150 triplets recorded

A directB automaticC forced

Accuracy: share of items correct

not measured (0 labeled items)

Evaluator overhead: items per minute (items / summed wall time)

Time consumed: total wall time

Headline (complete triplets)

accuracy	not measured (0 labeled items)
triplets	150
wall_A_median_s	3.058
wall_A_p90_s	5.279
wall_A_p95_s	6.174
jev_usd_A	0
called_jev_A	0
called_jev_share_A	0
wall_B_median_s	2.907
wall_B_p90_s	4.86
wall_B_p95_s	8.092
jev_usd_B	0
called_jev_B	0
called_jev_share_B	0
wall_C_median_s	13.95
wall_C_p90_s	21.01
wall_C_p95_s	28.67
jev_usd_C	0.005969
called_jev_C	150
called_jev_share_C	1
wall_b_minus_a_median_s	-0.1034
wall_b_minus_a_p90_s	1.034
wall_b_minus_a_p95_s	2.035
wall_b_minus_a_sign_p	0.1205
wall_c_minus_a_median_s	10.43
wall_c_minus_a_p90_s	15.99
wall_c_minus_a_p95_s	21.56
wall_c_minus_a_sign_p	1.587e-41

Runs

run	tool	status	use gate	answer	correct	wall s	Jev calls
classify-01.A	jev_classify	ok	n/a (arm has no Jev)	billing	False	2.72	0
classify-01.B	jev_classify	ok	did not call Jev	billing	False	2.45	0
classify-01.C	jev_classify	ok	used Jev	billing	False	11.31	1
classify-02.A	jev_classify	ok	n/a (arm has no Jev)	account	False	2.19	0
classify-02.B	jev_classify	ok	did not call Jev	account	False	1.67	0
classify-02.C	jev_classify	ok	used Jev	account	False	12.29	1
classify-03.A	jev_classify	ok	n/a (arm has no Jev)	shipping	False	1.59	0
classify-03.B	jev_classify	ok	did not call Jev	shipping	False	2.53	0
classify-03.C	jev_classify	ok	used Jev	shipping	False	11.77	1
classify-04.A	jev_classify	ok	n/a (arm has no Jev)	billing	False	2.71	0
classify-04.B	jev_classify	ok	did not call Jev	billing	False	2.43	0
classify-04.C	jev_classify	ok	used Jev	billing	False	16.82	1
classify-05.A	jev_classify	ok	n/a (arm has no Jev)	product_defect	False	2.55	0
classify-05.B	jev_classify	ok	did not call Jev	product_defect	False	2.76	0
classify-05.C	jev_classify	ok	used Jev	product_defect	False	13.72	1
classify-06.A	jev_classify	ok	n/a (arm has no Jev)	test_failure	False	2.36	0
classify-06.B	jev_classify	ok	did not call Jev	test_failure	False	2.69	0
classify-06.C	jev_classify	ok	used Jev	test_failure	False	12.68	1
classify-07.A	jev_classify	ok	n/a (arm has no Jev)	flaky	False	2.14	0
classify-07.B	jev_classify	ok	did not call Jev	flaky	False	2.28	0
classify-07.C	jev_classify	ok	used Jev	flaky	False	10.55	1
classify-08.A	jev_classify	ok	n/a (arm has no Jev)	config	False	3.58	0
classify-08.B	jev_classify	ok	did not call Jev	config	False	2.44	0
classify-08.C	jev_classify	ok	used Jev	config	False	12.08	1
classify-09.A	jev_classify	ok	n/a (arm has no Jev)	infra	False	2.72	0
classify-09.B	jev_classify	ok	did not call Jev	infra	False	3.21	0
classify-09.C	jev_classify	ok	used Jev	infra	False	13.13	1
classify-10.A	jev_classify	ok	n/a (arm has no Jev)	config	False	2.59	0
classify-10.B	jev_classify	ok	did not call Jev	config	False	2.57	0
classify-10.C	jev_classify	ok	used Jev	config	False	13.04	1
classify-11.A	jev_classify	ok	n/a (arm has no Jev)	docs_only	False	2.47	0
classify-11.B	jev_classify	ok	did not call Jev	docs_only	False	2.79	0
classify-11.C	jev_classify	ok	used Jev	docs_only	False	9.93	1
classify-12.A	jev_classify	ok	n/a (arm has no Jev)	breaking_change	False	3.56	0
classify-12.B	jev_classify	ok	did not call Jev	breaking_change	False	3.46	0
classify-12.C	jev_classify	ok	used Jev	breaking_change	False	14.12	1
classify-13.A	jev_classify	ok	n/a (arm has no Jev)	security	False	3.18	0
classify-13.B	jev_classify	ok	did not call Jev	security	False	2.59	0
classify-13.C	jev_classify	ok	used Jev	security	False	13.18	1
classify-14.A	jev_classify	ok	n/a (arm has no Jev)	bugfix	False	2.35	0
classify-14.B	jev_classify	ok	did not call Jev	bugfix	False	3.37	0
classify-14.C	jev_classify	ok	used Jev	bugfix	False	16.03	1
classify-15.A	jev_classify	ok	n/a (arm has no Jev)	security	False	3.05	0
classify-15.B	jev_classify	ok	did not call Jev	security	False	3.72	0
classify-15.C	jev_classify	ok	used Jev	security	False	16.23	1
compare-01.A	jev_compare	ok	n/a (arm has no Jev)	same_fact	False	3.07	0
compare-01.B	jev_compare	ok	did not call Jev	same_fact	False	2.92	0
compare-01.C	jev_compare	ok	used Jev	same_fact	False	8.70	1
compare-02.A	jev_compare	ok	n/a (arm has no Jev)	contradicts	False	2.93	0
compare-02.B	jev_compare	ok	did not call Jev	contradicts	False	4.38	0
compare-02.C	jev_compare	ok	used Jev	contradicts	False	12.44	1
compare-03.A	jev_compare	ok	n/a (arm has no Jev)	contradicts	False	3.57	0
compare-03.B	jev_compare	ok	did not call Jev	contradicts	False	3.83	0
compare-03.C	jev_compare	ok	used Jev	contradicts	False	11.67	1
compare-04.A	jev_compare	ok	n/a (arm has no Jev)	different_facts	False	1.91	0
compare-04.B	jev_compare	ok	did not call Jev	different_facts	False	3.02	0
compare-04.C	jev_compare	ok	used Jev	different_facts	False	9.50	1
compare-05.A	jev_compare	ok	n/a (arm has no Jev)	same_fact	False	2.77	0
compare-05.B	jev_compare	ok	did not call Jev	same_fact	False	2.17	0
compare-05.C	jev_compare	ok	used Jev	same_fact	False	12.56	1
compare-06.A	jev_compare	ok	n/a (arm has no Jev)	same_fact	False	2.00	0
compare-06.B	jev_compare	ok	did not call Jev	same_fact	False	1.79	0
compare-06.C	jev_compare	ok	used Jev	same_fact	False	12.24	1
compare-07.A	jev_compare	ok	n/a (arm has no Jev)	contradicts	False	3.17	0
compare-07.B	jev_compare	ok	did not call Jev	contradicts	False	2.43	0
compare-07.C	jev_compare	ok	used Jev	contradicts	False	12.64	1
compare-08.A	jev_compare	ok	n/a (arm has no Jev)	different_facts	False	2.54	0
compare-08.B	jev_compare	ok	did not call Jev	different_facts	False	2.39	0
compare-08.C	jev_compare	ok	used Jev	different_facts	False	9.75	1
compare-09.A	jev_compare	ok	n/a (arm has no Jev)	contradicts	False	2.51	0
compare-09.B	jev_compare	ok	did not call Jev	contradicts	False	1.50	0
compare-09.C	jev_compare	ok	used Jev	contradicts	False	9.15	1
compare-10.A	jev_compare	ok	n/a (arm has no Jev)	same_fact	False	2.16	0
compare-10.B	jev_compare	ok	did not call Jev	same_fact	False	1.79	0
compare-10.C	jev_compare	ok	used Jev	same_fact	False	8.51	1
compare-11.A	jev_compare	ok	n/a (arm has no Jev)	same_fact	False	2.53	0
compare-11.B	jev_compare	ok	did not call Jev	same_fact	False	2.37	0
compare-11.C	jev_compare	ok	used Jev	same_fact	False	12.59	1
compare-12.A	jev_compare	ok	n/a (arm has no Jev)	contradicts	False	2.48	0
compare-12.B	jev_compare	ok	did not call Jev	contradicts	False	3.81	0
compare-12.C	jev_compare	ok	used Jev	contradicts	False	12.84	1
compare-13.A	jev_compare	ok	n/a (arm has no Jev)	different_facts	False	2.91	0
compare-13.B	jev_compare	ok	did not call Jev	different_facts	False	2.36	0
compare-13.C	jev_compare	ok	used Jev	different_facts	False	13.82	1
compare-14.A	jev_compare	ok	n/a (arm has no Jev)	different_facts	False	2.75	0
compare-14.B	jev_compare	ok	did not call Jev	different_facts	False	2.15	0
compare-14.C	jev_compare	ok	used Jev	different_facts	False	12.45	1
compare-15.A	jev_compare	ok	n/a (arm has no Jev)	different_facts	False	2.19	0
compare-15.B	jev_compare	ok	did not call Jev	different_facts	False	2.81	0
compare-15.C	jev_compare	ok	used Jev	different_facts	False	11.00	1
decide-01.A	jev_decide	ok	n/a (arm has no Jev)	ship_full	False	5.28	0
decide-01.B	jev_decide	ok	did not call Jev	ship_full	False	4.81	0
decide-01.C	jev_decide	ok	used Jev	ship_full	False	16.80	1
decide-02.A	jev_decide	ok	n/a (arm has no Jev)	ship_flagged	False	3.43	0
decide-02.B	jev_decide	ok	did not call Jev	ship_flagged	False	4.47	0
decide-02.C	jev_decide	ok	used Jev	ship_flagged	False	12.20	1
decide-03.A	jev_decide	ok	n/a (arm has no Jev)	escape	False	42.08	0
decide-03.B	jev_decide	ok	did not call Jev	escape	False	17.28	0
decide-03.C	jev_decide	ok	used Jev	escape	False	30.95	1
decide-04.A	jev_decide	ok	n/a (arm has no Jev)	delay	False	5.19	0
decide-04.B	jev_decide	ok	did not call Jev	delay	False	5.07	0
decide-04.C	jev_decide	ok	used Jev	delay	False	15.07	1
decide-05.A	jev_decide	ok	n/a (arm has no Jev)	escape	False	27.26	0
decide-05.B	jev_decide	ok	did not call Jev	escape	False	5.94	0
decide-05.C	jev_decide	ok	used Jev	escape	False	66.03	1
decide-06.A	jev_decide	ok	n/a (arm has no Jev)	credit_20	False	3.56	0
decide-06.B	jev_decide	ok	did not call Jev	credit_20	False	2.81	0
decide-06.C	jev_decide	ok	used Jev	credit_20	False	15.53	1
decide-07.A	jev_decide	ok	n/a (arm has no Jev)	no_remedy	False	5.30	0
decide-07.B	jev_decide	ok	did not call Jev	no_remedy	False	4.87	0
decide-07.C	jev_decide	ok	used Jev	credit_20	False	52.04	1
decide-08.A	jev_decide	ok	n/a (arm has no Jev)	full_refund	False	3.34	0
decide-08.B	jev_decide	ok	did not call Jev	full_refund	False	3.28	0
decide-08.C	jev_decide	ok	used Jev	full_refund	False	13.90	1
decide-09.A	jev_decide	ok	n/a (arm has no Jev)	damage_claim	False	2.55	0
decide-09.B	jev_decide	ok	did not call Jev	damage_claim	False	2.63	0
decide-09.C	jev_decide	ok	used Jev	damage_claim	False	14.57	1
decide-10.A	jev_decide	ok	n/a (arm has no Jev)	no_remedy	False	5.40	0
decide-10.B	jev_decide	ok	did not call Jev	no_remedy	False	6.04	0
decide-10.C	jev_decide	ok	used Jev	escape	False	16.57	1
decide-11.A	jev_decide	ok	n/a (arm has no Jev)	revert_commit	False	3.75	0
decide-11.B	jev_decide	ok	did not call Jev	revert_commit	False	3.97	0
decide-11.C	jev_decide	ok	used Jev	revert_commit	False	15.60	1
decide-12.A	jev_decide	ok	n/a (arm has no Jev)	rerun	False	4.34	0
decide-12.B	jev_decide	ok	did not call Jev	rerun	False	2.17	0
decide-12.C	jev_decide	ok	used Jev	rerun	False	11.90	1
decide-13.A	jev_decide	ok	n/a (arm has no Jev)	escape	False	4.76	0
decide-13.B	jev_decide	ok	did not call Jev	escape	False	7.66	0
decide-13.C	jev_decide	ok	used Jev	escape	False	14.98	1
decide-14.A	jev_decide	ok	n/a (arm has no Jev)	fix_forward	False	2.86	0
decide-14.B	jev_decide	ok	did not call Jev	fix_forward	False	3.69	0
decide-14.C	jev_decide	ok	used Jev	fix_forward	False	13.82	1
decide-15.A	jev_decide	ok	n/a (arm has no Jev)	escape	False	5.55	0
decide-15.B	jev_decide	ok	did not call Jev	escape	False	4.32	0
decide-15.C	jev_decide	ok	used Jev	escape	False	17.26	1
extract-01.A	jev_extract	ok	n/a (arm has no Jev)	$5,153.50	False	3.05	0
extract-01.B	jev_extract	ok	did not call Jev	$5,153.50	False	3.93	0
extract-01.C	jev_extract	ok	used Jev	$5,153.50	False	15.38	1
extract-02.A	jev_extract	ok	n/a (arm has no Jev)	not stated	False	3.74	0
extract-02.B	jev_extract	ok	did not call Jev	not stated	False	2.96	0
extract-02.C	jev_extract	ok	used Jev	not stated	False	14.93	1
extract-03.A	jev_extract	ok	n/a (arm has no Jev)	$1,020.00	False	2.66	0
extract-03.B	jev_extract	ok	did not call Jev	$1,020.00	False	2.62	0
extract-03.C	jev_extract	ok	used Jev	$1,020.00	False	18.24	1
extract-04.A	jev_extract	ok	n/a (arm has no Jev)	NF-2026-08102	False	2.18	0
extract-04.B	jev_extract	ok	did not call Jev	NF-2026-08102	False	2.45	0
extract-04.C	jev_extract	ok	used Jev	NF-2026-08102	False	9.25	1
extract-05.A	jev_extract	ok	n/a (arm has no Jev)	not stated	False	3.49	0
extract-05.B	jev_extract	ok	did not call Jev	not stated	False	2.99	0
extract-05.C	jev_extract	ok	used Jev	not stated	False	14.58	1
extract-06.A	jev_extract	ok	n/a (arm has no Jev)	shopkit-4.4.0-rc2	False	2.64	0
extract-06.B	jev_extract	ok	did not call Jev	shopkit-4.4.0-rc2	False	3.36	0
extract-06.C	jev_extract	ok	used Jev	shopkit-4.4.0-rc2	False	18.75	1
extract-07.A	jev_extract	ok	n/a (arm has no Jev)	not stated	False	4.25	0
extract-07.B	jev_extract	ok	did not call Jev	not stated	False	4.82	0
extract-07.C	jev_extract	ok	used Jev	not stated	False	16.58	1
extract-08.A	jev_extract	ok	n/a (arm has no Jev)	1	False	2.86	0
extract-08.B	jev_extract	ok	did not call Jev	1	False	2.63	0
extract-08.C	jev_extract	ok	used Jev	1	False	15.53	1
extract-09.A	jev_extract	ok	n/a (arm has no Jev)	deps-v2-a41c	False	2.84	0
extract-09.B	jev_extract	ok	did not call Jev	deps-v2-a41c	False	8.44	0
extract-09.C	jev_extract	ok	used Jev	deps-v2-a41c	False	12.79	1
extract-10.A	jev_extract	ok	n/a (arm has no Jev)	not stated	False	2.66	0
extract-10.B	jev_extract	ok	did not call Jev	not stated	False	2.83	0
extract-10.C	jev_extract	ok	used Jev	not stated	False	15.24	1
extract-11.A	jev_extract	ok	n/a (arm has no Jev)	BKT-2291-0475	False	2.87	0
extract-11.B	jev_extract	ok	did not call Jev	BKT-2291-0475	False	2.34	0
extract-11.C	jev_extract	ok	used Jev	BKT-2291-0475	False	14.03	1
extract-12.A	jev_extract	ok	n/a (arm has no Jev)	SO-778120	False	3.17	0
extract-12.B	jev_extract	ok	did not call Jev	SO-778120	False	2.33	0
extract-12.C	jev_extract	ok	used Jev	SO-778120	False	10.31	1
extract-13.A	jev_extract	ok	n/a (arm has no Jev)	2026-02-14	False	1.98	0
extract-13.B	jev_extract	ok	did not call Jev	2026-02-14	False	2.63	0
extract-13.C	jev_extract	ok	used Jev	2026-02-14	False	15.06	1
extract-14.A	jev_extract	ok	n/a (arm has no Jev)	2026-09-12	False	3.10	0
extract-14.B	jev_extract	ok	did not call Jev	2026-09-12	False	2.17	0
extract-14.C	jev_extract	ok	used Jev	2026-09-12	False	14.15	1
extract-15.A	jev_extract	ok	n/a (arm has no Jev)	not stated	False	2.62	0
extract-15.B	jev_extract	ok	did not call Jev	not stated	False	2.47	0
extract-15.C	jev_extract	ok	used Jev	not stated	False	12.82	1
find-01.A	jev_find	ok	n/a (arm has no Jev)	ret-final	False	2.40	0
find-01.B	jev_find	ok	did not call Jev	ret-final	False	2.24	0
find-01.C	jev_find	ok	used Jev	ret-final	False	13.61	1
find-02.A	jev_find	ok	n/a (arm has no Jev)	price-drop	False	2.69	0
find-02.B	jev_find	ok	did not call Jev	price-drop	False	2.12	0
find-02.C	jev_find	ok	used Jev	price-drop	False	16.83	1
find-03.A	jev_find	ok	n/a (arm has no Jev)	ship-intl	False	1.59	0
find-03.B	jev_find	ok	did not call Jev	ship-intl	False	2.97	0
find-03.C	jev_find	ok	used Jev	ship-intl	False	11.31	1
find-04.A	jev_find	ok	n/a (arm has no Jev)	pay-methods	False	2.65	0
find-04.B	jev_find	ok	did not call Jev	pay-methods	False	2.16	0
find-04.C	jev_find	ok	used Jev	pay-methods	False	12.15	1
find-05.A	jev_find	ok	n/a (arm has no Jev)	none	False	2.26	0
find-05.B	jev_find	ok	did not call Jev	none	False	2.17	0
find-05.C	jev_find	ok	used Jev	none	False	10.65	1
find-06.A	jev_find	ok	n/a (arm has no Jev)	registry-401	False	3.83	0
find-06.B	jev_find	ok	did not call Jev	registry-401	False	4.86	0
find-06.C	jev_find	ok	used Jev	registry-401	False	13.29	1
find-07.A	jev_find	ok	n/a (arm has no Jev)	flaky-quarantine	False	8.01	0
find-07.B	jev_find	ok	did not call Jev	none	False	3.26	0
find-07.C	jev_find	ok	used Jev	none	False	17.01	1
find-08.A	jev_find	ok	n/a (arm has no Jev)	job-timeouts	False	3.06	0
find-08.B	jev_find	ok	did not call Jev	job-timeouts	False	2.98	0
find-08.C	jev_find	ok	used Jev	job-timeouts	False	15.39	1
find-09.A	jev_find	ok	n/a (arm has no Jev)	none	False	3.16	0
find-09.B	jev_find	ok	did not call Jev	none	False	1.82	0
find-09.C	jev_find	ok	used Jev	none	False	16.80	1
find-10.A	jev_find	ok	n/a (arm has no Jev)	cache-bust	False	1.92	0
find-10.B	jev_find	ok	did not call Jev	cache-bust	False	1.38	0
find-10.C	jev_find	ok	used Jev	cache-bust	False	13.49	1
find-11.A	jev_find	ok	n/a (arm has no Jev)	v3.3.0	False	3.12	0
find-11.B	jev_find	ok	did not call Jev	v3.3.0	False	2.84	0
find-11.C	jev_find	ok	used Jev	v3.3.0	False	14.43	1
find-12.A	jev_find	ok	n/a (arm has no Jev)	v3.2.1	False	2.02	0
find-12.B	jev_find	ok	did not call Jev	v3.2.1	False	3.05	0
find-12.C	jev_find	ok	used Jev	v3.2.1	False	19.92	2
find-13.A	jev_find	ok	n/a (arm has no Jev)	none	False	2.53	0
find-13.B	jev_find	ok	did not call Jev	none	False	2.20	0
find-13.C	jev_find	ok	used Jev	none	False	12.30	1
find-14.A	jev_find	ok	n/a (arm has no Jev)	v3.1.1	False	1.48	0
find-14.B	jev_find	ok	did not call Jev	v3.1.1	False	1.92	0
find-14.C	jev_find	ok	used Jev	v3.1.1	False	12.76	1
find-15.A	jev_find	ok	n/a (arm has no Jev)	none	False	2.19	0
find-15.B	jev_find	ok	did not call Jev	none	False	2.55	0
find-15.C	jev_find	ok	used Jev	none	False	18.17	1
gate-01.A	jev_gate	ok	n/a (arm has no Jev)	accept	False	40.53	0
gate-01.B	jev_gate	ok	did not call Jev	accept	False	8.73	0
gate-01.C	jev_gate	ok	used Jev	reject	False	25.17	1
gate-02.A	jev_gate	ok	n/a (arm has no Jev)	reject	False	2.62	0
gate-02.B	jev_gate	ok	did not call Jev	reject	False	2.39	0
gate-02.C	jev_gate	ok	used Jev	reject	False	13.21	1
gate-03.A	jev_gate	ok	n/a (arm has no Jev)	reject	False	4.14	0
gate-03.B	jev_gate	ok	did not call Jev	reject	False	4.17	0
gate-03.C	jev_gate	ok	used Jev	reject	False	14.38	1
gate-04.A	jev_gate	ok	n/a (arm has no Jev)	accept	False	2.89	0
gate-04.B	jev_gate	ok	did not call Jev	accept	False	2.89	0
gate-04.C	jev_gate	ok	used Jev	accept	False	16.28	1
gate-05.A	jev_gate	ok	n/a (arm has no Jev)	accept	False	6.02	0
gate-05.B	jev_gate	ok	did not call Jev	accept	False	5.17	0
gate-05.C	jev_gate	ok	used Jev	accept	False	14.39	1
gate-06.A	jev_gate	ok	n/a (arm has no Jev)	reject	False	4.41	0
gate-06.B	jev_gate	ok	did not call Jev	reject	False	2.50	0
gate-06.C	jev_gate	ok	used Jev	reject	False	14.34	1
gate-07.A	jev_gate	ok	n/a (arm has no Jev)	accept	False	3.56	0
gate-07.B	jev_gate	ok	did not call Jev	accept	False	9.53	0
gate-07.C	jev_gate	ok	used Jev	accept	False	69.59	1
gate-08.A	jev_gate	ok	n/a (arm has no Jev)	reject	False	4.70	0
gate-08.B	jev_gate	ok	did not call Jev	reject	False	4.32	0
gate-08.C	jev_gate	ok	used Jev	reject	False	13.65	1
gate-09.A	jev_gate	ok	n/a (arm has no Jev)	reject	False	4.49	0
gate-09.B	jev_gate	ok	did not call Jev	reject	False	3.41	0
gate-09.C	jev_gate	ok	used Jev	reject	False	14.42	1
gate-10.A	jev_gate	ok	n/a (arm has no Jev)	reject	False	3.47	0
gate-10.B	jev_gate	ok	did not call Jev	reject	False	4.25	0
gate-10.C	jev_gate	ok	used Jev	reject	False	16.39	1
gate-11.A	jev_gate	ok	n/a (arm has no Jev)	accept	False	4.07	0
gate-11.B	jev_gate	ok	did not call Jev	accept	False	24.55	0
gate-11.C	jev_gate	ok	used Jev	reject	False	44.08	1
gate-12.A	jev_gate	ok	n/a (arm has no Jev)	reject	False	4.85	0
gate-12.B	jev_gate	ok	did not call Jev	reject	False	3.12	0
gate-12.C	jev_gate	ok	used Jev	reject	False	16.36	1
gate-13.A	jev_gate	ok	n/a (arm has no Jev)	accept	False	2.85	0
gate-13.B	jev_gate	ok	did not call Jev	accept	False	2.33	0
gate-13.C	jev_gate	ok	used Jev	accept	False	14.45	1
gate-14.A	jev_gate	ok	n/a (arm has no Jev)	reject	False	3.98	0
gate-14.B	jev_gate	ok	did not call Jev	reject	False	4.54	0
gate-14.C	jev_gate	ok	used Jev	reject	False	14.47	1
gate-15.A	jev_gate	ok	n/a (arm has no Jev)	accept	False	6.20	0
gate-15.B	jev_gate	ok	did not call Jev	reject	False	93.19	0
gate-15.C	jev_gate	ok	used Jev	accept	False	21.76	1
rerank-01.A	jev_rerank	ok	n/a (arm has no Jev)	['kb-e5', 'kb-e3', 'kb-warranty', 'kb-descale', 'kb-firmware', 'kb-wifi']	False	3.48	0
rerank-01.B	jev_rerank	ok	did not call Jev	['kb-e5', 'kb-warranty', 'kb-e3', 'kb-descale', 'kb-wifi', 'kb-firmware']	False	4.43	0
rerank-01.C	jev_rerank	ok	used Jev	['kb-e5', 'kb-descale', 'kb-warranty', 'kb-firmware', 'kb-e3', 'kb-wifi']	False	52.77	1
rerank-02.A	jev_rerank	ok	n/a (arm has no Jev)	['kb-warranty', 'kb-descale', 'kb-wifi', 'kb-firmware', 'kb-app-login', 'kb-colors']	False	5.71	0
rerank-02.B	jev_rerank	ok	did not call Jev	['kb-warranty', 'kb-descale', 'kb-firmware', 'kb-wifi', 'kb-app-login', 'kb-colors']	False	4.06	0
rerank-02.C	jev_rerank	ok	used Jev	['kb-warranty', 'kb-firmware', 'kb-descale', 'kb-wifi', 'kb-app-login', 'kb-colors']	False	20.51	1
rerank-03.A	jev_rerank	ok	n/a (arm has no Jev)	['ret-2026', 'ret-2025', 'ret-business', 'kb-refund-timing', 'kb-warranty']	False	5.11	0
rerank-03.B	jev_rerank	ok	did not call Jev	['ret-2026', 'ret-2025', 'ret-business', 'kb-refund-timing', 'kb-warranty']	False	4.33	0
rerank-03.C	jev_rerank	ok	used Jev	['ret-2026', 'ret-business', 'ret-2025', 'kb-refund-timing', 'kb-warranty']	False	10.05	1
rerank-04.A	jev_rerank	ok	n/a (arm has no Jev)	['kb-keepwarm', 'kb-formula', 'kb-schedule', 'kb-descale', 'kb-firmware']	False	3.68	0
rerank-04.B	jev_rerank	ok	did not call Jev	['kb-keepwarm', 'kb-formula', 'kb-schedule', 'kb-descale', 'kb-firmware']	False	18.72	0
rerank-04.C	jev_rerank	ok	used Jev	['kb-keepwarm', 'kb-formula', 'kb-schedule', 'kb-descale', 'kb-firmware']	False	14.71	1
rerank-05.A	jev_rerank	ok	n/a (arm has no Jev)	['kb-wifi', 'kb-app-login', 'kb-reset', 'kb-firmware', 'kb-e3']	False	7.31	0
rerank-05.B	jev_rerank	ok	did not call Jev	['kb-wifi', 'kb-reset', 'kb-app-login', 'kb-firmware', 'kb-e3']	False	3.33	0
rerank-05.C	jev_rerank	ok	used Jev	['kb-wifi', 'kb-app-login', 'kb-reset', 'kb-firmware', 'kb-e3']	False	14.06	1
rerank-06.A	jev_rerank	ok	n/a (arm has no Jev)	['rb-oom', 'rb-timeout', 'rb-disk', 'rb-cache', 'rb-flaky']	False	3.42	0
rerank-06.B	jev_rerank	ok	did not call Jev	['rb-oom', 'rb-timeout', 'rb-disk', 'rb-cache', 'rb-flaky']	False	2.33	0
rerank-06.C	jev_rerank	ok	used Jev	['rb-oom', 'rb-disk', 'rb-timeout', 'rb-flaky', 'rb-cache']	False	13.16	1
rerank-07.A	jev_rerank	ok	n/a (arm has no Jev)	['rb-perm', 'rb-cache', 'rb-registry', 'rb-disk', 'rb-flaky', 'rb-manual']	False	3.63	0
rerank-07.B	jev_rerank	ok	did not call Jev	['rb-perm', 'rb-cache', 'rb-registry', 'rb-disk', 'rb-flaky', 'rb-manual']	False	3.66	0
rerank-07.C	jev_rerank	ok	used Jev	['rb-perm', 'rb-cache', 'rb-disk', 'rb-flaky', 'rb-registry', 'rb-manual']	False	13.55	1
rerank-08.A	jev_rerank	ok	n/a (arm has no Jev)	['rb-registry', 'rb-perm', 'rb-cache', 'rb-disk', 'rb-oom']	False	4.27	0
rerank-08.B	jev_rerank	ok	did not call Jev	['rb-registry', 'rb-cache', 'rb-disk', 'rb-oom', 'rb-perm']	False	3.22	0
rerank-08.C	jev_rerank	ok	used Jev	['rb-registry', 'rb-cache', 'rb-disk', 'rb-perm', 'rb-oom']	False	12.99	1
rerank-09.A	jev_rerank	ok	n/a (arm has no Jev)	['rb-flaky', 'rb-timeout', 'rb-oom', 'rb-cache', 'rb-disk', 'rb-manual']	False	9.30	0
rerank-09.B	jev_rerank	ok	did not call Jev	['rb-flaky', 'rb-timeout', 'rb-oom', 'rb-cache', 'rb-disk', 'rb-manual']	False	3.87	0
rerank-09.C	jev_rerank	ok	used Jev	['rb-flaky', 'rb-timeout', 'rb-cache', 'rb-disk', 'rb-oom', 'rb-manual']	False	16.62	1
rerank-10.A	jev_rerank	ok	n/a (arm has no Jev)	['rb-timeout', 'rb-manual', 'rb-oom', 'rb-disk', 'rb-registry']	False	3.03	0
rerank-10.B	jev_rerank	ok	did not call Jev	['rb-timeout', 'rb-manual', 'rb-oom', 'rb-disk', 'rb-registry']	False	4.86	0
rerank-10.C	jev_rerank	ok	used Jev	['rb-timeout', 'rb-manual', 'rb-oom', 'rb-disk', 'rb-registry']	False	15.61	1
rerank-11.A	jev_rerank	ok	n/a (arm has no Jev)	['v4.4.0', 'v4.3.0', 'v4.3.1', 'v4.2.0', 'v4.1.2', 'v4.0.0']	False	5.52	0
rerank-11.B	jev_rerank	ok	did not call Jev	['v4.4.0', 'v4.3.0', 'v4.3.1', 'v4.2.0', 'v4.1.2', 'v4.0.0']	False	3.26	0
rerank-11.C	jev_rerank	ok	used Jev	['v4.4.0', 'v4.3.0', 'v4.3.1', 'v4.2.0', 'v4.1.2', 'v4.0.0']	False	21.72	1
rerank-12.A	jev_rerank	ok	n/a (arm has no Jev)	['v4.1.2', 'v4.1.0', 'v4.2.0', 'v4.3.0', 'v4.0.0']	False	3.10	0
rerank-12.B	jev_rerank	ok	did not call Jev	['v4.1.2', 'v4.1.0', 'v4.2.0', 'v4.3.0', 'v4.0.0']	False	2.85	0
rerank-12.C	jev_rerank	ok	used Jev	['v4.1.2', 'v4.1.0', 'v4.2.0', 'v4.3.0', 'v4.0.0']	False	14.43	1
rerank-13.A	jev_rerank	ok	n/a (arm has no Jev)	['v4.1.0', 'v4.1.2', 'v4.2.1', 'v4.4.0', 'v4.0.0']	False	2.74	0
rerank-13.B	jev_rerank	ok	did not call Jev	['v4.1.0', 'v4.1.2', 'v4.2.1', 'v4.4.0', 'v4.0.0']	False	4.94	0
rerank-13.C	jev_rerank	ok	used Jev	['v4.1.0', 'v4.1.2', 'v4.0.0', 'v4.2.1', 'v4.4.0']	False	12.51	1
rerank-14.A	jev_rerank	ok	n/a (arm has no Jev)	['v4.2.1', 'v4.2.0', 'v4.3.0', 'v4.3.1', 'v4.4.0', 'v4.1.2']	False	2.83	0
rerank-14.B	jev_rerank	ok	did not call Jev	['v4.2.1', 'v4.2.0', 'v4.4.0', 'v4.3.0', 'v4.3.1', 'v4.1.2']	False	3.03	0
rerank-14.C	jev_rerank	ok	used Jev	['v4.2.1', 'v4.4.0', 'v4.3.1', 'v4.2.0', 'v4.3.0', 'v4.1.2']	False	11.76	1
rerank-15.A	jev_rerank	ok	n/a (arm has no Jev)	['v4.3.1', 'v4.4.0', 'v4.3.0', 'v4.1.0', 'v4.2.0', 'v4.1.2', 'v4.0.0']	False	3.66	0
rerank-15.B	jev_rerank	ok	did not call Jev	['v4.3.1', 'v4.3.0', 'v4.4.0', 'v4.2.0', 'v4.1.2', 'v4.1.0', 'v4.0.0']	False	4.03	0
rerank-15.C	jev_rerank	ok	used Jev	['v4.3.1', 'v4.3.0', 'v4.4.0', 'v4.0.0', 'v4.1.2', 'v4.2.0', 'v4.1.0']	False	14.78	1
review-01.A	jev_review	ok	n/a (arm has no Jev)	do_not_apply	False	4.45	0
review-01.B	jev_review	ok	did not call Jev	do_not_apply	False	3.63	0
review-01.C	jev_review	ok	used Jev	do_not_apply	False	14.82	1
review-02.A	jev_review	ok	n/a (arm has no Jev)	apply	False	7.90	0
review-02.B	jev_review	ok	did not call Jev	apply	False	14.80	0
review-02.C	jev_review	ok	used Jev	do_not_apply	False	25.89	1
review-03.A	jev_review	ok	n/a (arm has no Jev)	do_not_apply	False	5.22	0
review-03.B	jev_review	ok	did not call Jev	do_not_apply	False	3.20	0
review-03.C	jev_review	ok	used Jev	do_not_apply	False	22.49	1
review-04.A	jev_review	ok	n/a (arm has no Jev)	apply	False	4.10	0
review-04.B	jev_review	ok	did not call Jev	apply	False	4.44	0
review-04.C	jev_review	ok	used Jev	apply	False	15.75	1
review-05.A	jev_review	ok	n/a (arm has no Jev)	do_not_apply	False	2.79	0
review-05.B	jev_review	ok	did not call Jev	do_not_apply	False	2.54	0
review-05.C	jev_review	ok	used Jev	do_not_apply	False	9.63	1
review-06.A	jev_review	ok	n/a (arm has no Jev)	do_not_apply	False	3.59	0
review-06.B	jev_review	ok	did not call Jev	do_not_apply	False	3.71	0
review-06.C	jev_review	ok	used Jev	do_not_apply	False	17.70	1
review-07.A	jev_review	ok	n/a (arm has no Jev)	apply	False	2.56	0
review-07.B	jev_review	ok	did not call Jev	apply	False	2.64	0
review-07.C	jev_review	ok	used Jev	apply	False	41.75	3
review-08.A	jev_review	ok	n/a (arm has no Jev)	do_not_apply	False	6.14	0
review-08.B	jev_review	ok	did not call Jev	do_not_apply	False	3.68	0
review-08.C	jev_review	ok	used Jev	do_not_apply	False	13.32	1
review-09.A	jev_review	ok	n/a (arm has no Jev)	apply	False	2.81	0
review-09.B	jev_review	ok	did not call Jev	apply	False	2.03	0
review-09.C	jev_review	ok	used Jev	apply	False	16.32	1
review-10.A	jev_review	ok	n/a (arm has no Jev)	do_not_apply	False	3.27	0
review-10.B	jev_review	ok	did not call Jev	do_not_apply	False	3.07	0
review-10.C	jev_review	ok	used Jev	do_not_apply	False	8.92	1
review-11.A	jev_review	ok	n/a (arm has no Jev)	do_not_apply	False	3.06	0
review-11.B	jev_review	ok	did not call Jev	do_not_apply	False	2.16	0
review-11.C	jev_review	ok	used Jev	do_not_apply	False	13.18	1
review-12.A	jev_review	ok	n/a (arm has no Jev)	apply	False	3.06	0
review-12.B	jev_review	ok	did not call Jev	apply	False	2.15	0
review-12.C	jev_review	ok	used Jev	apply	False	33.55	2
review-13.A	jev_review	ok	n/a (arm has no Jev)	do_not_apply	False	3.11	0
review-13.B	jev_review	ok	did not call Jev	do_not_apply	False	3.43	0
review-13.C	jev_review	ok	used Jev	do_not_apply	False	9.93	1
review-14.A	jev_review	ok	n/a (arm has no Jev)	apply	False	2.75	0
review-14.B	jev_review	ok	did not call Jev	apply	False	2.75	0
review-14.C	jev_review	ok	used Jev	apply	False	20.94	1
review-15.A	jev_review	ok	n/a (arm has no Jev)	apply	False	3.42	0
review-15.B	jev_review	ok	did not call Jev	apply	False	4.62	0
review-15.C	jev_review	ok	used Jev	do_not_apply	False	12.23	1
screen-01.A	jev_screen	ok	n/a (arm has no Jev)	clean	False	2.37	0
screen-01.B	jev_screen	ok	did not call Jev	clean	False	2.81	0
screen-01.C	jev_screen	ok	used Jev	clean	False	13.17	1
screen-02.A	jev_screen	ok	n/a (arm has no Jev)	injection	False	2.99	0
screen-02.B	jev_screen	ok	did not call Jev	injection	False	2.68	0
screen-02.C	jev_screen	ok	no successful Jev call (only errored or refused calls)	injection	False	24.15	3
screen-03.A	jev_screen	ok	n/a (arm has no Jev)	clean	False	3.99	0
screen-03.B	jev_screen	ok	did not call Jev	clean	False	2.87	0
screen-03.C	jev_screen	ok	used Jev	clean	False	11.13	1
screen-04.A	jev_screen	ok	n/a (arm has no Jev)	injection	False	3.06	0
screen-04.B	jev_screen	ok	did not call Jev	injection	False	3.03	0
screen-04.C	jev_screen	ok	used Jev	injection	False	12.90	1
screen-05.A	jev_screen	ok	n/a (arm has no Jev)	clean	False	2.68	0
screen-05.B	jev_screen	ok	did not call Jev	clean	False	2.70	0
screen-05.C	jev_screen	ok	used Jev	clean	False	13.91	1
screen-06.A	jev_screen	ok	n/a (arm has no Jev)	injection	False	2.28	0
screen-06.B	jev_screen	ok	did not call Jev	injection	False	3.10	0
screen-06.C	jev_screen	ok	used Jev	injection	False	11.50	1
screen-07.A	jev_screen	ok	n/a (arm has no Jev)	clean	False	3.36	0
screen-07.B	jev_screen	ok	did not call Jev	clean	False	2.61	0
screen-07.C	jev_screen	ok	used Jev	clean	False	13.98	1
screen-08.A	jev_screen	ok	n/a (arm has no Jev)	injection	False	2.27	0
screen-08.B	jev_screen	ok	did not call Jev	injection	False	2.47	0
screen-08.C	jev_screen	ok	used Jev	injection	False	16.31	1
screen-09.A	jev_screen	ok	n/a (arm has no Jev)	clean	False	3.80	0
screen-09.B	jev_screen	ok	did not call Jev	clean	False	2.79	0
screen-09.C	jev_screen	ok	used Jev	clean	False	11.21	1
screen-10.A	jev_screen	ok	n/a (arm has no Jev)	injection	False	3.37	0
screen-10.B	jev_screen	ok	did not call Jev	injection	False	3.45	0
screen-10.C	jev_screen	ok	used Jev	injection	False	13.31	1
screen-11.A	jev_screen	ok	n/a (arm has no Jev)	clean	False	3.14	0
screen-11.B	jev_screen	ok	did not call Jev	clean	False	2.44	0
screen-11.C	jev_screen	ok	used Jev	clean	False	16.24	1
screen-12.A	jev_screen	ok	n/a (arm has no Jev)	injection	False	3.40	0
screen-12.B	jev_screen	ok	did not call Jev	injection	False	2.04	0
screen-12.C	jev_screen	ok	used Jev	injection	False	13.26	1
screen-13.A	jev_screen	ok	n/a (arm has no Jev)	clean	False	2.80	0
screen-13.B	jev_screen	ok	did not call Jev	clean	False	3.17	0
screen-13.C	jev_screen	ok	used Jev	clean	False	12.10	1
screen-14.A	jev_screen	ok	n/a (arm has no Jev)	injection	False	2.68	0
screen-14.B	jev_screen	ok	did not call Jev	injection	False	3.08	0
screen-14.C	jev_screen	ok	used Jev	injection	False	10.84	1
screen-15.A	jev_screen	ok	n/a (arm has no Jev)	clean	False	2.40	0
screen-15.B	jev_screen	ok	did not call Jev	clean	False	2.82	0
screen-15.C	jev_screen	ok	used Jev	clean	False	12.92	1
verify-01.A	jev_verify	ok	n/a (arm has no Jev)	contradicted	False	2.19	0
verify-01.B	jev_verify	ok	did not call Jev	contradicted	False	1.47	0
verify-01.C	jev_verify	ok	used Jev	contradicted	False	11.38	1
verify-02.A	jev_verify	ok	n/a (arm has no Jev)	verified	False	2.86	0
verify-02.B	jev_verify	ok	did not call Jev	verified	False	2.79	0
verify-02.C	jev_verify	ok	used Jev	verified	False	11.64	1
verify-03.A	jev_verify	ok	n/a (arm has no Jev)	contradicted	False	2.07	0
verify-03.B	jev_verify	ok	did not call Jev	contradicted	False	1.96	0
verify-03.C	jev_verify	ok	used Jev	contradicted	False	10.47	1
verify-04.A	jev_verify	ok	n/a (arm has no Jev)	verified	False	3.34	0
verify-04.B	jev_verify	ok	did not call Jev	verified	False	1.76	0
verify-04.C	jev_verify	ok	used Jev	verified	False	16.25	1
verify-05.A	jev_verify	ok	n/a (arm has no Jev)	unsupported	False	2.27	0
verify-05.B	jev_verify	ok	did not call Jev	unsupported	False	2.39	0
verify-05.C	jev_verify	ok	used Jev	unsupported	False	15.59	1
verify-06.A	jev_verify	ok	n/a (arm has no Jev)	verified	False	3.48	0
verify-06.B	jev_verify	ok	did not call Jev	verified	False	3.18	0
verify-06.C	jev_verify	ok	used Jev	verified	False	17.21	1
verify-07.A	jev_verify	ok	n/a (arm has no Jev)	contradicted	False	2.40	0
verify-07.B	jev_verify	ok	did not call Jev	contradicted	False	2.57	0
verify-07.C	jev_verify	ok	used Jev	contradicted	False	11.68	1
verify-08.A	jev_verify	ok	n/a (arm has no Jev)	unsupported	False	3.06	0
verify-08.B	jev_verify	ok	did not call Jev	unsupported	False	3.94	0
verify-08.C	jev_verify	ok	used Jev	unsupported	False	16.66	1
verify-09.A	jev_verify	ok	n/a (arm has no Jev)	verified	False	2.76	0
verify-09.B	jev_verify	ok	did not call Jev	verified	False	2.12	0
verify-09.C	jev_verify	ok	used Jev	verified	False	12.29	1
verify-10.A	jev_verify	ok	n/a (arm has no Jev)	unsupported	False	3.23	0
verify-10.B	jev_verify	ok	did not call Jev	unsupported	False	3.21	0
verify-10.C	jev_verify	ok	used Jev	unsupported	False	13.41	1
verify-11.A	jev_verify	ok	n/a (arm has no Jev)	contradicted	False	2.75	0
verify-11.B	jev_verify	ok	did not call Jev	contradicted	False	2.77	0
verify-11.C	jev_verify	ok	used Jev	contradicted	False	11.01	1
verify-12.A	jev_verify	ok	n/a (arm has no Jev)	unsupported	False	3.07	0
verify-12.B	jev_verify	ok	did not call Jev	unsupported	False	2.84	0
verify-12.C	jev_verify	ok	used Jev	unsupported	False	13.49	1
verify-13.A	jev_verify	ok	n/a (arm has no Jev)	contradicted	False	3.14	0
verify-13.B	jev_verify	ok	did not call Jev	contradicted	False	2.55	0
verify-13.C	jev_verify	ok	used Jev	verified	False	25.03	2
verify-14.A	jev_verify	ok	n/a (arm has no Jev)	unsupported	False	3.05	0
verify-14.B	jev_verify	ok	did not call Jev	unsupported	False	3.54	0
verify-14.C	jev_verify	ok	used Jev	unsupported	False	16.95	1
verify-15.A	jev_verify	ok	n/a (arm has no Jev)	verified	False	2.88	0
verify-15.B	jev_verify	ok	did not call Jev	verified	False	3.13	0
verify-15.C	jev_verify	ok	used Jev	verified	False	16.14	1