== dev
   misfold   0.01  jev: caught 163/195 (83.6%), flagged 4/195 (2.1%)  openai: caught 85/195 (43.6%), flagged 2/195 (1.0%)
   misfold   0.02  jev: caught 175/195 (89.7%), flagged 5/195 (2.6%)  openai: caught 160/195 (82.1%), flagged 6/195 (3.1%)
   misfold   0.05  jev: caught 184/195 (94.4%), flagged 8/195 (4.1%)  openai: caught 185/195 (94.9%), flagged 13/195 (6.7%)
   misfold    0.1  jev: caught 186/195 (95.4%), flagged 16/195 (8.2%)  openai: caught 188/195 (96.4%), flagged 19/195 (9.7%)
   misfold   0.15  jev: caught 188/195 (96.4%), flagged 18/195 (9.2%)  openai: caught 190/195 (97.4%), flagged 22/195 (11.3%)
   misfold    0.2  jev: caught 190/195 (97.4%), flagged 22/195 (11.3%)  openai: caught 190/195 (97.4%), flagged 28/195 (14.4%)
   misfold   0.25  jev: caught 191/195 (97.9%), flagged 24/195 (12.3%)  openai: caught 193/195 (99.0%), flagged 35/195 (17.9%)
   misfold    0.3  jev: caught 191/195 (97.9%), flagged 25/195 (12.8%)  openai: caught 193/195 (99.0%), flagged 42/195 (21.5%)
   misfold   0.35  jev: caught 191/195 (97.9%), flagged 28/195 (14.4%)  openai: caught 193/195 (99.0%), flagged 45/195 (23.1%)
   misfold    0.4  jev: caught 192/195 (98.5%), flagged 30/195 (15.4%)  openai: caught 194/195 (99.5%), flagged 49/195 (25.1%)
   misfold    0.5  jev: caught 194/195 (99.5%), flagged 37/195 (19.0%)  openai: caught 195/195 (100.0%), flagged 58/195 (29.7%)
   owner      0.5  jev: answered 180/200 (90.0%), right 160/180 (88.9%)  openai: answered 157/200 (78.5%), right 139/157 (88.5%)
   owner      0.6  jev: answered 165/200 (82.5%), right 152/165 (92.1%)  openai: answered 138/200 (69.0%), right 123/138 (89.1%)
   owner      0.7  jev: answered 154/200 (77.0%), right 144/154 (93.5%)  openai: answered 118/200 (59.0%), right 109/118 (92.4%)
   owner      0.8  jev: answered 136/200 (68.0%), right 129/136 (94.9%)  openai: answered 96/200 (48.0%), right 92/96 (95.8%)
   owner      0.9  jev: answered 111/200 (55.5%), right 109/111 (98.2%)  openai: answered 70/200 (35.0%), right 68/70 (97.1%)
   owner     0.95  jev: answered 99/200 (49.5%), right 99/99 (100.0%)  openai: answered 47/200 (23.5%), right 46/47 (97.9%)
   owner     0.99  jev: answered 73/200 (36.5%), right 73/73 (100.0%)  openai: answered 19/200 (9.5%), right 19/19 (100.0%)
   owner      1.0  jev: answered 50/200 (25.0%), right 50/50 (100.0%)  openai: answered 12/200 (6.0%), right 12/12 (100.0%)
   sentence  0.01  jev: caught 0/79 (0.0%), flagged 0/79 (0.0%)       openai: caught 28/79 (35.4%), flagged 0/79 (0.0%)
   sentence  0.02  jev: caught 0/79 (0.0%), flagged 0/79 (0.0%)       openai: caught 35/79 (44.3%), flagged 0/79 (0.0%)
   sentence  0.05  jev: caught 8/79 (10.1%), flagged 0/79 (0.0%)      openai: caught 40/79 (50.6%), flagged 0/79 (0.0%)
   sentence   0.1  jev: caught 28/79 (35.4%), flagged 0/79 (0.0%)     openai: caught 47/79 (59.5%), flagged 0/79 (0.0%)
   sentence  0.15  jev: caught 44/79 (55.7%), flagged 0/79 (0.0%)     openai: caught 54/79 (68.4%), flagged 0/79 (0.0%)
   sentence   0.2  jev: caught 53/79 (67.1%), flagged 1/79 (1.3%)     openai: caught 59/79 (74.7%), flagged 1/79 (1.3%)
   sentence  0.25  jev: caught 59/79 (74.7%), flagged 2/79 (2.5%)     openai: caught 60/79 (75.9%), flagged 1/79 (1.3%)
   sentence   0.3  jev: caught 63/79 (79.7%), flagged 2/79 (2.5%)     openai: caught 64/79 (81.0%), flagged 1/79 (1.3%)
   sentence  0.35  jev: caught 67/79 (84.8%), flagged 2/79 (2.5%)     openai: caught 67/79 (84.8%), flagged 2/79 (2.5%)
   sentence   0.4  jev: caught 68/79 (86.1%), flagged 6/79 (7.6%)     openai: caught 67/79 (84.8%), flagged 2/79 (2.5%)
   sentence   0.5  jev: caught 74/79 (93.7%), flagged 8/79 (10.1%)    openai: caught 70/79 (88.6%), flagged 4/79 (5.1%)
   flow      0.01  jev: caught 0/80 (0.0%), flagged 0/80 (0.0%)       openai: caught 20/80 (25.0%), flagged 1/80 (1.2%)
   flow      0.02  jev: caught 0/80 (0.0%), flagged 0/80 (0.0%)       openai: caught 27/80 (33.8%), flagged 1/80 (1.2%)
   flow      0.05  jev: caught 9/80 (11.2%), flagged 0/80 (0.0%)      openai: caught 37/80 (46.2%), flagged 2/80 (2.5%)
   flow       0.1  jev: caught 34/80 (42.5%), flagged 0/80 (0.0%)     openai: caught 44/80 (55.0%), flagged 7/80 (8.8%)
   flow      0.15  jev: caught 42/80 (52.5%), flagged 2/80 (2.5%)     openai: caught 53/80 (66.2%), flagged 12/80 (15.0%)
   flow       0.2  jev: caught 53/80 (66.2%), flagged 2/80 (2.5%)     openai: caught 62/80 (77.5%), flagged 17/80 (21.2%)
   flow      0.25  jev: caught 60/80 (75.0%), flagged 6/80 (7.5%)     openai: caught 66/80 (82.5%), flagged 22/80 (27.5%)
   flow       0.3  jev: caught 63/80 (78.8%), flagged 11/80 (13.8%)   openai: caught 71/80 (88.8%), flagged 35/80 (43.8%)
   flow      0.35  jev: caught 67/80 (83.8%), flagged 15/80 (18.8%)   openai: caught 74/80 (92.5%), flagged 40/80 (50.0%)
   flow       0.4  jev: caught 72/80 (90.0%), flagged 22/80 (27.5%)   openai: caught 75/80 (93.8%), flagged 42/80 (52.5%)
   flow       0.5  jev: caught 73/80 (91.2%), flagged 39/80 (48.8%)   openai: caught 77/80 (96.2%), flagged 49/80 (61.3%)
   governs    0.5  jev: found 143/177 (80.8%), suggested 57/354 (16.1%)  openai: found 122/177 (68.9%), suggested 60/354 (16.9%)
   governs    0.6  jev: found 123/177 (69.5%), suggested 29/354 (8.2%)  openai: found 119/177 (67.2%), suggested 56/354 (15.8%)
   governs    0.7  jev: found 94/177 (53.1%), suggested 13/354 (3.7%)  openai: found 111/177 (62.7%), suggested 50/354 (14.1%)
   governs    0.8  jev: found 54/177 (30.5%), suggested 5/354 (1.4%)  openai: found 100/177 (56.5%), suggested 42/354 (11.9%)
   governs    0.9  jev: found 15/177 (8.5%), suggested 0/354 (0.0%)   openai: found 74/177 (41.8%), suggested 20/354 (5.6%)
   governs   0.95  jev: found 0/177 (0.0%), suggested 0/354 (0.0%)    openai: found 59/177 (33.3%), suggested 12/354 (3.4%)
   governs   0.99  jev: found 0/177 (0.0%), suggested 0/354 (0.0%)    openai: found 35/177 (19.8%), suggested 4/354 (1.1%)
   governs    1.0  jev: found 0/177 (0.0%), suggested 0/354 (0.0%)    openai: found 18/177 (10.2%), suggested 3/354 (0.8%)
   plan       0.5  jev: covered 0.69, extra 0.16                      openai: covered 0.73, extra 0.10
   plan       0.3  jev: covered 0.79, extra 0.26                      openai: covered 0.80, extra 0.16
   plan       0.2  jev: covered 0.82, extra 0.42                      openai: covered 0.82, extra 0.28
   plan       0.1  jev: covered 0.86, extra 0.69                      openai: covered 0.85, extra 0.54
   plan      0.05  jev: covered 0.86, extra 1.15                      openai: covered 0.89, extra 0.82
   plan      0.02  jev: covered 0.88, extra 1.91                      openai: covered 0.92, extra 1.71
   triage       -  jev: top-1 64/80 (80.0%)                           openai: top-1 66/80 (82.5%)
== holdout
   misfold   0.01  jev: caught 72/96 (75.0%), flagged 1/96 (1.0%)     openai: caught 23/96 (24.0%), flagged 0/96 (0.0%)
   misfold   0.02  jev: caught 80/96 (83.3%), flagged 1/96 (1.0%)     openai: caught 59/96 (61.5%), flagged 1/96 (1.0%)
   misfold   0.05  jev: caught 87/96 (90.6%), flagged 1/96 (1.0%)     openai: caught 85/96 (88.5%), flagged 2/96 (2.1%)
   misfold    0.1  jev: caught 91/96 (94.8%), flagged 2/96 (2.1%)     openai: caught 90/96 (93.8%), flagged 5/96 (5.2%)
   misfold   0.15  jev: caught 94/96 (97.9%), flagged 3/96 (3.1%)     openai: caught 91/96 (94.8%), flagged 6/96 (6.2%)
   misfold    0.2  jev: caught 95/96 (99.0%), flagged 4/96 (4.2%)     openai: caught 92/96 (95.8%), flagged 9/96 (9.4%)
   misfold   0.25  jev: caught 95/96 (99.0%), flagged 4/96 (4.2%)     openai: caught 93/96 (96.9%), flagged 11/96 (11.5%)
   misfold    0.3  jev: caught 96/96 (100.0%), flagged 4/96 (4.2%)    openai: caught 95/96 (99.0%), flagged 12/96 (12.5%)
   misfold   0.35  jev: caught 96/96 (100.0%), flagged 6/96 (6.2%)    openai: caught 95/96 (99.0%), flagged 15/96 (15.6%)
   misfold    0.4  jev: caught 96/96 (100.0%), flagged 7/96 (7.3%)    openai: caught 95/96 (99.0%), flagged 21/96 (21.9%)
   misfold    0.5  jev: caught 96/96 (100.0%), flagged 8/96 (8.3%)    openai: caught 95/96 (99.0%), flagged 32/96 (33.3%)
   owner      0.5  jev: answered 102/106 (96.2%), right 96/102 (94.1%)  openai: answered 71/106 (67.0%), right 67/71 (94.4%)
   owner      0.6  jev: answered 96/106 (90.6%), right 91/96 (94.8%)  openai: answered 64/106 (60.4%), right 62/64 (96.9%)
   owner      0.7  jev: answered 85/106 (80.2%), right 83/85 (97.6%)  openai: answered 57/106 (53.8%), right 56/57 (98.2%)
   owner      0.8  jev: answered 81/106 (76.4%), right 79/81 (97.5%)  openai: answered 48/106 (45.3%), right 48/48 (100.0%)
   owner      0.9  jev: answered 64/106 (60.4%), right 64/64 (100.0%)  openai: answered 24/106 (22.6%), right 24/24 (100.0%)
   owner     0.95  jev: answered 53/106 (50.0%), right 53/53 (100.0%)  openai: answered 13/106 (12.3%), right 13/13 (100.0%)
   owner     0.99  jev: answered 39/106 (36.8%), right 39/39 (100.0%)  openai: answered 5/106 (4.7%), right 5/5 (100.0%)
   owner      1.0  jev: answered 27/106 (25.5%), right 27/27 (100.0%)  openai: answered 3/106 (2.8%), right 3/3 (100.0%)
   sentence  0.01  jev: caught 0/44 (0.0%), flagged 0/44 (0.0%)       openai: caught 8/44 (18.2%), flagged 0/44 (0.0%)
   sentence  0.02  jev: caught 0/44 (0.0%), flagged 0/44 (0.0%)       openai: caught 8/44 (18.2%), flagged 0/44 (0.0%)
   sentence  0.05  jev: caught 2/44 (4.5%), flagged 0/44 (0.0%)       openai: caught 11/44 (25.0%), flagged 0/44 (0.0%)
   sentence   0.1  jev: caught 8/44 (18.2%), flagged 0/44 (0.0%)      openai: caught 17/44 (38.6%), flagged 0/44 (0.0%)
   sentence  0.15  jev: caught 17/44 (38.6%), flagged 0/44 (0.0%)     openai: caught 19/44 (43.2%), flagged 0/44 (0.0%)
   sentence   0.2  jev: caught 27/44 (61.4%), flagged 1/44 (2.3%)     openai: caught 24/44 (54.5%), flagged 0/44 (0.0%)
   sentence  0.25  jev: caught 32/44 (72.7%), flagged 1/44 (2.3%)     openai: caught 28/44 (63.6%), flagged 0/44 (0.0%)
   sentence   0.3  jev: caught 40/44 (90.9%), flagged 2/44 (4.5%)     openai: caught 33/44 (75.0%), flagged 0/44 (0.0%)
   sentence  0.35  jev: caught 41/44 (93.2%), flagged 2/44 (4.5%)     openai: caught 35/44 (79.5%), flagged 0/44 (0.0%)
   sentence   0.4  jev: caught 41/44 (93.2%), flagged 4/44 (9.1%)     openai: caught 38/44 (86.4%), flagged 1/44 (2.3%)
   sentence   0.5  jev: caught 41/44 (93.2%), flagged 6/44 (13.6%)    openai: caught 41/44 (93.2%), flagged 2/44 (4.5%)
   flow      0.01  jev: caught 0/48 (0.0%), flagged 0/48 (0.0%)       openai: caught 9/48 (18.8%), flagged 0/48 (0.0%)
   flow      0.02  jev: caught 0/48 (0.0%), flagged 0/48 (0.0%)       openai: caught 14/48 (29.2%), flagged 1/48 (2.1%)
   flow      0.05  jev: caught 1/48 (2.1%), flagged 0/48 (0.0%)       openai: caught 27/48 (56.2%), flagged 4/48 (8.3%)
   flow       0.1  jev: caught 13/48 (27.1%), flagged 1/48 (2.1%)     openai: caught 33/48 (68.8%), flagged 5/48 (10.4%)
   flow      0.15  jev: caught 21/48 (43.8%), flagged 2/48 (4.2%)     openai: caught 36/48 (75.0%), flagged 8/48 (16.7%)
   flow       0.2  jev: caught 26/48 (54.2%), flagged 2/48 (4.2%)     openai: caught 39/48 (81.2%), flagged 11/48 (22.9%)
   flow      0.25  jev: caught 36/48 (75.0%), flagged 6/48 (12.5%)    openai: caught 41/48 (85.4%), flagged 15/48 (31.2%)
   flow       0.3  jev: caught 40/48 (83.3%), flagged 7/48 (14.6%)    openai: caught 42/48 (87.5%), flagged 19/48 (39.6%)
   flow      0.35  jev: caught 43/48 (89.6%), flagged 13/48 (27.1%)   openai: caught 44/48 (91.7%), flagged 25/48 (52.1%)
   flow       0.4  jev: caught 44/48 (91.7%), flagged 19/48 (39.6%)   openai: caught 46/48 (95.8%), flagged 27/48 (56.2%)
   flow       0.5  jev: caught 45/48 (93.8%), flagged 27/48 (56.2%)   openai: caught 46/48 (95.8%), flagged 32/48 (66.7%)
   governs    0.5  jev: found 49/58 (84.5%), suggested 33/116 (28.4%)  openai: found 53/58 (91.4%), suggested 62/116 (53.4%)
   governs    0.6  jev: found 39/58 (67.2%), suggested 20/116 (17.2%)  openai: found 51/58 (87.9%), suggested 55/116 (47.4%)
   governs    0.7  jev: found 32/58 (55.2%), suggested 7/116 (6.0%)   openai: found 48/58 (82.8%), suggested 48/116 (41.4%)
   governs    0.8  jev: found 22/58 (37.9%), suggested 1/116 (0.9%)   openai: found 45/58 (77.6%), suggested 43/116 (37.1%)
   governs    0.9  jev: found 11/58 (19.0%), suggested 0/116 (0.0%)   openai: found 33/58 (56.9%), suggested 28/116 (24.1%)
   governs   0.95  jev: found 0/58 (0.0%), suggested 0/116 (0.0%)     openai: found 23/58 (39.7%), suggested 14/116 (12.1%)
   governs   0.99  jev: found 0/58 (0.0%), suggested 0/116 (0.0%)     openai: found 10/58 (17.2%), suggested 7/116 (6.0%)
   governs    1.0  jev: found 0/58 (0.0%), suggested 0/116 (0.0%)     openai: found 7/58 (12.1%), suggested 4/116 (3.4%)
   plan       0.5  jev: covered 0.46, extra 0.13                      openai: covered 0.39, extra 0.13
   plan       0.3  jev: covered 0.58, extra 0.33                      openai: covered 0.47, extra 0.33
   plan       0.2  jev: covered 0.63, extra 0.36                      openai: covered 0.61, extra 0.51
   plan       0.1  jev: covered 0.66, extra 0.59                      openai: covered 0.66, extra 0.85
   plan      0.05  jev: covered 0.71, extra 0.90                      openai: covered 0.74, extra 1.18
   plan      0.02  jev: covered 0.74, extra 1.31                      openai: covered 0.82, extra 2.44
   triage       -  jev: top-1 31/39 (79.5%)                           openai: top-1 27/39 (69.2%)
