Evaluation report — expense-triage

Customer

northwind

Suite

expense-triage

This run

staging — 2026-08-27T09:45:39.141911+00:00

Reference

staging — 2026-08-27T09:45:38.769454+00:00

Code version

b5870a655c776d75f8be518796bad8acf02fb08a-dirty — uncommitted changes were present, so this run cannot be reproduced from the repository

Did it get worse? Yes

1 check got worse compared with the reference. Every case could be judged. No case is suspended. The configuration is the same as the reference.

regressed 1
errored 0
new 0
missing 0
improved 0
unchanged 21

Overall

Measure	Result	What happened	Why
`precision`	0.700000 / 0.600000	20 counted · 0 suspended · 0 not judged	precision 0.700000 = 7/10 (20 counted, 0 suspended, 0 could not be judged)
`accuracy`	0.800000 / 0.700000	20 counted · 0 suspended · 0 not judged	accuracy 0.800000 = 16/20 (20 counted, 0 suspended, 0 could not be judged)

What got worse (1)

Case	Check	What happened	Why
`hotel_over`	`agrees_with_mark`	Went from passing to failing (0.800000 → 0.400000).	mean of 5 samples (0.000000, 0.000000, 1.000000, 0.000000, 1.000000)

What could not be judged (0)

Nothing in this section.

What is set aside (0)

Nothing in this section.

What was added or removed (0)

Nothing in this section.

What got better (0)

Nothing in this section.

What stayed the same (21)

Case	Check	What happened	Why
`amount_over_cap`	`agrees_with_mark`	Score unchanged at 1.000000.	mean of 5 samples (1.000000, 1.000000, 1.000000, 1.000000, 1.000000)
`book_technical`	`agrees_with_mark`	Score unchanged at 1.000000.	mean of 5 samples (1.000000, 1.000000, 1.000000, 1.000000, 1.000000)
`coffee_two`	`agrees_with_mark`	Score unchanged at 1.000000.	mean of 5 samples (1.000000, 1.000000, 1.000000, 1.000000, 1.000000)
`courier_urgent`	`agrees_with_mark`	Score unchanged at 1.000000.	mean of 5 samples (1.000000, 1.000000, 1.000000, 1.000000, 1.000000)
`dinner_client`	`agrees_with_mark`	Score unchanged at 0.400000.	mean of 5 samples (0.000000, 0.000000, 0.000000, 1.000000, 1.000000)
`dinner_solo`	`agrees_with_mark`	Score unchanged at 1.000000.	mean of 5 samples (1.000000, 1.000000, 1.000000, 1.000000, 1.000000)
`duplicate_claim`	`agrees_with_mark`	Score unchanged at 1.000000.	mean of 5 samples (1.000000, 1.000000, 1.000000, 1.000000, 1.000000)
`hotel_capped`	`agrees_with_mark`	Score unchanged at 1.000000.	mean of 5 samples (1.000000, 1.000000, 1.000000, 1.000000, 1.000000)
`late_taxi`	`agrees_with_mark`	Score unchanged at 1.000000.	mean of 5 samples (1.000000, 1.000000, 1.000000, 1.000000, 1.000000)
`lunch_team`	`agrees_with_mark`	Score unchanged at 0.600000.	mean of 5 samples (1.000000, 0.000000, 1.000000, 0.000000, 1.000000)
`meal_no_note`	`agrees_with_mark`	Score unchanged at 1.000000.	mean of 5 samples (1.000000, 1.000000, 1.000000, 1.000000, 1.000000)
`monitor_home`	`agrees_with_mark`	Score unchanged at 1.000000.	mean of 5 samples (1.000000, 1.000000, 1.000000, 1.000000, 1.000000)
`no_receipt`	`agrees_with_mark`	Score unchanged at 1.000000.	mean of 5 samples (1.000000, 1.000000, 1.000000, 1.000000, 1.000000)
`parking_airport`	`agrees_with_mark`	Score unchanged at 0.000000.	mean of 5 samples (0.000000, 0.000000, 0.000000, 0.000000, 0.000000)
`software_seat`	`agrees_with_mark`	Score unchanged at 0.000000.	mean of 5 samples (0.000000, 0.000000, 0.000000, 0.000000, 0.000000)
`taxi_long`	`agrees_with_mark`	Score unchanged at 0.600000.	mean of 5 samples (0.000000, 1.000000, 0.000000, 1.000000, 1.000000)
`taxi_short`	`agrees_with_mark`	Score unchanged at 1.000000.	mean of 5 samples (1.000000, 1.000000, 1.000000, 1.000000, 1.000000)
`train_standard`	`agrees_with_mark`	Score unchanged at 1.000000.	mean of 5 samples (1.000000, 1.000000, 1.000000, 1.000000, 1.000000)
`weekend_bar`	`agrees_with_mark`	Score unchanged at 1.000000.	mean of 5 samples (1.000000, 1.000000, 1.000000, 1.000000, 1.000000)
	`precision`	Score unchanged at 0.700000.	precision 0.700000 = 7/10 (20 counted, 0 suspended, 0 could not be judged)
	`accuracy`	Score unchanged at 0.800000.	accuracy 0.800000 = 16/20 (20 counted, 0 suspended, 0 could not be judged)