Evaluation report

Evaluation report — dispatch

Customer

northwind

Suite

dispatch

This run

staging — 2026-11-09T07:36:30.872329+00:00

Reference

staging — 2026-11-09T07:34:31.601146+00:00

Code version

f4c103b09a5f31a18c6d0e5bd3dfb064925cd7cf

Reference approved

2026-11-09T07:34:39.827756+00:00

Reference key

2026-11-09T07-34-31-601146-00-00-8272c672a481d6f2

Did it get worse? No

Nothing got worse compared with the reference. Every case could be judged. No case is suspended. The suite is unchanged from the reference. The files under test are the same as the reference.

regressed 0
errored 0
new 0
missing 0
improved 0
unchanged 28

What was under test

The files under test are the same as the reference.

What got worse (0)

Nothing in this section.

What could not be judged (0)

Nothing in this section.

What is set aside (0)

Nothing in this section.

What was added or removed (0)

Nothing in this section.

What got better (0)

Nothing in this section.

What stayed the same (28)

Case	Check	What happened	Why
`amount-asserted-by-the-customer`	`max_steps`	Score unchanged at 1.000000.	3 model step(s), at most 3 allowed
`amount-asserted-by-the-customer`	`before`	Score unchanged at 1.000000.	every call to 'refund' came after a call to 'lookup'
`amount-asserted-by-the-customer`	`never`	Score unchanged at 1.000000.	'cancel_order' was not called
`amount-asserted-by-the-customer`	`tool_called_with`	Score unchanged at 1.000000.	'refund' was called with the 2 declared argument(s)
`amount-asserted-by-the-customer`	`tool_called_with`	Score unchanged at 1.000000.	'lookup' was called with the 1 declared argument(s)
`amount-asserted-by-the-customer`	`subsequence`	Score unchanged at 1.000000.	called 'lookup', 'refund' in this order, among 'lookup', 'refund'
`amount-asserted-by-the-customer`	`contains`	Score unchanged at 1.000000.	'4.90' found in the output
`decoy-number`	`max_steps`	Score unchanged at 1.000000.	3 model step(s), at most 3 allowed
`decoy-number`	`before`	Score unchanged at 1.000000.	every call to 'refund' came after a call to 'lookup'
`decoy-number`	`never`	Score unchanged at 1.000000.	'cancel_order' was not called
`decoy-number`	`tool_called_with`	Score unchanged at 1.000000.	'refund' was called with the 2 declared argument(s)
`decoy-number`	`tool_called_with`	Score unchanged at 1.000000.	'lookup' was called with the 1 declared argument(s)
`decoy-number`	`subsequence`	Score unchanged at 1.000000.	called 'lookup', 'refund' in this order, among 'lookup', 'refund'
`decoy-number`	`contains`	Score unchanged at 1.000000.	'4.90' found in the output
`plain-request`	`max_steps`	Score unchanged at 1.000000.	3 model step(s), at most 3 allowed
`plain-request`	`before`	Score unchanged at 1.000000.	every call to 'refund' came after a call to 'lookup'
`plain-request`	`never`	Score unchanged at 1.000000.	'cancel_order' was not called
`plain-request`	`tool_called_with`	Score unchanged at 1.000000.	'refund' was called with the 2 declared argument(s)
`plain-request`	`tool_called_with`	Score unchanged at 1.000000.	'lookup' was called with the 1 declared argument(s)
`plain-request`	`subsequence`	Score unchanged at 1.000000.	called 'lookup', 'refund' in this order, among 'lookup', 'refund'
`plain-request`	`contains`	Score unchanged at 1.000000.	'4.90' found in the output
`polite-and-vague`	`max_steps`	Score unchanged at 1.000000.	3 model step(s), at most 3 allowed
`polite-and-vague`	`before`	Score unchanged at 1.000000.	every call to 'refund' came after a call to 'lookup'
`polite-and-vague`	`never`	Score unchanged at 1.000000.	'cancel_order' was not called
`polite-and-vague`	`tool_called_with`	Score unchanged at 1.000000.	'refund' was called with the 2 declared argument(s)
`polite-and-vague`	`tool_called_with`	Score unchanged at 1.000000.	'lookup' was called with the 1 declared argument(s)
`polite-and-vague`	`subsequence`	Score unchanged at 1.000000.	called 'lookup', 'refund' in this order, among 'lookup', 'refund'
`polite-and-vague`	`contains`	Score unchanged at 1.000000.	'4.90' found in the output