
######## Kimi-K2: 1000 runs, unparsable calls 0

== Toucan Kimi-K2 good: 541 successful, 0 failed runs, 7.7 s
   successful runs flagged   figured  39.0%  baseline  39.9%
   failed runs flagged       figured   n/a  baseline   n/a
   corruptions caught        figured 331/354  baseline 305
   substitutions caught      figured 89/289  baseline 7
   calls to unknown tools    successful runs 39  failed runs 0

== Toucan Kimi-K2 middle: 0 successful, 276 failed runs, 2.6 s
   successful runs flagged   figured   n/a  baseline   n/a
   failed runs flagged       figured  54.3%  baseline  57.6%
   corruptions caught        figured 0/0  baseline 0
   substitutions caught      figured 0/0  baseline 0
   calls to unknown tools    successful runs 0  failed runs 53

== Toucan Kimi-K2 poor: 0 successful, 183 failed runs, 1.6 s
   successful runs flagged   figured   n/a  baseline   n/a
   failed runs flagged       figured  47.0%  baseline  46.4%
   corruptions caught        figured 0/0  baseline 0
   substitutions caught      figured 0/0  baseline 0
   calls to unknown tools    successful runs 0  failed runs 63

######## Qwen3: 1000 runs, unparsable calls 0

== Toucan Qwen3 good: 210 successful, 0 failed runs, 4.8 s
   successful runs flagged   figured  48.6%  baseline  43.8%
   failed runs flagged       figured   n/a  baseline   n/a
   corruptions caught        figured 177/179  baseline 160
   substitutions caught      figured 39/134  baseline 0
   calls to unknown tools    successful runs 2  failed runs 0

== Toucan Qwen3 middle: 0 successful, 339 failed runs, 4.4 s
   successful runs flagged   figured   n/a  baseline   n/a
   failed runs flagged       figured  54.0%  baseline  50.4%
   corruptions caught        figured 0/0  baseline 0
   substitutions caught      figured 0/0  baseline 0
   calls to unknown tools    successful runs 0  failed runs 40

== Toucan Qwen3 poor: 0 successful, 451 failed runs, 4.7 s
   successful runs flagged   figured   n/a  baseline   n/a
   failed runs flagged       figured  45.9%  baseline  42.8%
   corruptions caught        figured 0/0  baseline 0
   substitutions caught      figured 0/0  baseline 0
   calls to unknown tools    successful runs 0  failed runs 22

######## OSS: 1000 runs, unparsable calls 0

== Toucan OSS good: 854 successful, 0 failed runs, 18.2 s
   successful runs flagged   figured  40.7%  baseline  40.3%
   failed runs flagged       figured   n/a  baseline   n/a
   corruptions caught        figured 607/627  baseline 546
   substitutions caught      figured 85/428  baseline 5
   calls to unknown tools    successful runs 96  failed runs 0

== Toucan OSS middle: 0 successful, 95 failed runs, 0.7 s
   successful runs flagged   figured   n/a  baseline   n/a
   failed runs flagged       figured  34.7%  baseline  38.9%
   corruptions caught        figured 0/0  baseline 0
   substitutions caught      figured 0/0  baseline 0
   calls to unknown tools    successful runs 0  failed runs 2

== Toucan OSS poor: 0 successful, 51 failed runs, 0.3 s
   successful runs flagged   figured   n/a  baseline   n/a
   failed runs flagged       figured  27.5%  baseline  37.3%
   corruptions caught        figured 0/0  baseline 0
   substitutions caught      figured 0/0  baseline 0
   calls to unknown tools    successful runs 0  failed runs 18

== speed: 10634 calls, before_call p50 46.7 µs, p99 930.9 µs
