Migration Report — gpt-4o-mini-2024-07-18 → claude-haiku-4-5

Corpus corpus · target claude-haiku-4-5 (anthropic) · generated 2026-06-16T04:55:51+00:00 · comparators: exact, fuzzy
100 compared (100 cached, 0 live) · 0 skipped · 0 errored

Summary

Comparator	Passed	Pass rate	Mean score
exact	22/100	22%	0.22
fuzzy	48/100	48%	0.65

Metric	Baseline	Target	Ratio
Output tokens	1,435	3,059	2.13×
Est. cost (anthropic-list+openai-list)	$0.001658	$0.020655	12.46×

By category

Cells are pass rate · mean score. Out tokens and Latency are target/baseline ratios.

Category	Prompts	exact	fuzzy	Out tokens	Latency	Cost (anthropic-list+openai-list)
classify	30	7% · 0.07	13% · 0.25	9.40×	–	25.16×
extract	30	57% · 0.57	93% · 0.95	1.42×	–	9.69×
rewrite	15	13% · 0.13	40% · 0.72	1.65×	–	10.45×
summarize	15	0% · 0.00	7% · 0.62	1.18×	–	8.60×
translate	10	10% · 0.10	90% · 0.90	1.68×	–	10.89×

Results

#	Prompt	Category	Baseline	exact	fuzzy	Out tok	Latency	Cost (anthropic-list+openai-list)	Cached
1	Extract as JSON {error_code, service} from this log line: '2026-06-11T09:14:02Z…	extract	gpt-4o-mini-2024-07-18	❌ 0.00	✅ 0.98	23→32	–	$0.000024→$0.000227	yes
2	Extract as JSON {amount, currency}: 'Your total comes to $1,249.99 including ta…	extract	gpt-4o-mini-2024-07-18	✅ 1.00	✅ 1.00	23→30	–	$0.000021→$0.000199	yes
3	Extract as JSON {checkin, checkout} as ISO dates: 'We'd like the room from Marc…	extract	gpt-4o-mini-2024-07-18	❌ 0.00	✅ 0.84	45→38	–	$0.000036→$0.00025	yes
4	Fix grammar and spelling, return only the corrected text: 'Me and him was going…	rewrite	gpt-4o-mini-2024-07-18	❌ 0.00	✅ 0.96	12→15	–	$0.000015→$0.000129	yes
5	Sentiment (positive/negative/neutral): 'Shipping was fast but the product broke…	classify	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.13	2→31	–	$0.000009→$0.000203	yes
6	Sentiment (positive/negative/neutral): 'Absolutely love it, exceeded every expe…	classify	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.52	1→10	–	$0.000008→$0.000096	yes
7	Is this about a delivery problem? (yes/no): 'The tracking number you sent does …	classify	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.02	2→43	–	$0.000009→$0.000263	yes
8	Summarize this policy in one sentence: 'Employees may work remotely up to three…	summarize	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.78	32→35	–	$0.00003→$0.000247	yes
9	Topic (billing/bug/feature_request/how_to): 'How do I add a teammate to my work…	classify	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.12	48→198	–	$0.000036→$0.001042	yes
10	Rewrite this error message to be user-friendly: 'ERR_CONN_5021: pool exhausted …	rewrite	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.44	19→28	–	$0.000019→$0.000194	yes
11	Summarize the complaint in one sentence: 'I ordered a blue medium jacket but re…	summarize	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.64	29→35	–	$0.000027→$0.000238	yes
12	Extract the contract end date as an ISO date: 'The agreement runs through 31 De…	extract	gpt-4o-mini-2024-07-18	✅ 1.00	✅ 1.00	6→10	–	$0.000012→$0.000104	yes
13	Remove the filler words: 'So basically we just kind of need to actually finaliz…	rewrite	gpt-4o-mini-2024-07-18	❌ 0.00	✅ 0.97	8→13	–	$0.000012→$0.000114	yes
14	Normalize this address into one line with proper capitalization: 'lagerstrasse …	rewrite	gpt-4o-mini-2024-07-18	✅ 1.00	✅ 1.00	12→18	–	$0.000015→$0.000142	yes
15	Turn this into a polite decline: 'No, we won't build that feature.'	rewrite	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.52	20→33	–	$0.000019→$0.000209	yes
16	Extract as JSON {temperature_c, sensor}: 'Sensor 7 in the cold room reads -18.5…	extract	gpt-4o-mini-2024-07-18	✅ 1.00	✅ 1.00	23→30	–	$0.000022→$0.000206	yes
17	Replace all personal data with [REDACTED]: 'Contact Anna Larsen at +45 22 11 33…	rewrite	gpt-4o-mini-2024-07-18	❌ 0.00	✅ 0.98	25→24	–	$0.000024→$0.000183	yes
18	Rewrite as a single bullet point: 'The meeting covered the hiring plan, the off…	rewrite	gpt-4o-mini-2024-07-18	❌ 0.00	✅ 0.91	15→19	–	$0.000017→$0.000147	yes
19	Extract as JSON {sku, quantity}: 'Please ship 12 units of SKU-44821 to the Hamb…	extract	gpt-4o-mini-2024-07-18	❌ 0.00	✅ 0.93	16→31	–	$0.000017→$0.000209	yes
20	Extract the IBAN: 'Please wire the deposit to IBAN DE89 3704 0044 0532 0130 00 …	extract	gpt-4o-mini-2024-07-18	❌ 0.00	✅ 0.90	16→11	–	$0.000019→$0.000116	yes
21	Route this support ticket (billing/technical/account/shipping): 'I can't reset …	classify	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.10	1→6	–	$0.000008→$0.000077	yes
22	Spam or not spam: 'Hi Sam, can we move our 2pm meeting to 3pm tomorrow?'	classify	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.16	3→23	–	$0.000009→$0.000166	yes
23	Intent (question/complaint/praise/refund_request): 'The blender stopped working…	classify	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.19	3→37	–	$0.00001→$0.00024	yes
24	Language of this text (ISO 639-1 code only): '¿Dónde está la estación de tren?'	classify	gpt-4o-mini-2024-07-18	✅ 1.00	✅ 1.00	1→4	–	$0.000008→$0.000075	yes
25	Extract as JSON {flight, gate}: 'Flight UA287 is now boarding at gate B14.'	extract	gpt-4o-mini-2024-07-18	✅ 1.00	✅ 1.00	22→29	–	$0.00002→$0.000192	yes
26	Topic (billing/bug/feature_request/how_to): 'It would be great if exports suppo…	classify	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.24	3→29	–	$0.000009→$0.000195	yes
27	Spam or not spam: 'Congratulations! You won a $1000 gift card, click here to cl…	classify	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.05	2→54	–	$0.000009→$0.000322	yes
28	Summarize this review in one sentence: 'I have used this laptop for three month…	summarize	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.71	24→30	–	$0.000025→$0.000224	yes
29	One-sentence executive summary: 'Q2 churn rose from 4% to 6%, driven by small-b…	summarize	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.61	34→31	–	$0.000031→$0.000224	yes
30	Extract the invoice number: 'Re: outstanding payment for invoice INV-2026-0457.'	extract	gpt-4o-mini-2024-07-18	✅ 1.00	✅ 1.00	7→11	–	$0.000011→$0.000102	yes
31	Extract all URLs as a JSON array: 'Docs moved to https://docs.example.com/v2; o…	extract	gpt-4o-mini-2024-07-18	❌ 0.00	✅ 0.98	21→36	–	$0.000021→$0.000242	yes
32	Priority (low/medium/high/urgent): 'Checkout intermittently fails for about 10%…	classify	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.08	3→86	–	$0.000009→$0.000481	yes
33	Extract as JSON {from, to} airport codes: 'Searching flights CPH to JFK for two…	extract	gpt-4o-mini-2024-07-18	✅ 1.00	✅ 1.00	22→29	–	$0.000021→$0.000195	yes
34	Extract as JSON {city, country}: 'Our HQ is in Lyon, France, with a satellite o…	extract	gpt-4o-mini-2024-07-18	❌ 0.00	✅ 0.98	33→55	–	$0.000028→$0.000326	yes
35	Summarize the risk in one sentence: 'While the merger promises cost savings thr…	summarize	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.46	20→25	–	$0.000021→$0.000185	yes
36	Does this text contain personal data? (yes/no): 'My SSN is 523-12-9876, please …	classify	gpt-4o-mini-2024-07-18	❌ 0.00	✅ 0.86	2→4	–	$0.00001→$0.000076	yes
37	Translate to Dutch: 'Your invoice is attached as a PDF.'	translate	gpt-4o-mini-2024-07-18	✅ 1.00	✅ 1.00	10→18	–	$0.000012→$0.00013	yes
38	Extract all dates as a JSON array of ISO dates: 'The invoice was issued 2026-03…	extract	gpt-4o-mini-2024-07-18	✅ 1.00	✅ 1.00	20→24	–	$0.000021→$0.000181	yes
39	Neutralize the tone for a status update: 'The vendor STILL has not fixed their …	rewrite	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.66	16→22	–	$0.000018→$0.000165	yes
40	Condense this email to one sentence: 'Following up on our call - attached is th…	summarize	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.76	22→37	–	$0.000024→$0.000256	yes
41	Churn risk (low/medium/high): 'We're evaluating alternatives because prices wen…	classify	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.00	2→106	–	$0.000008→$0.000579	yes
42	Summarize the incident impact in one sentence: 'The pricing bug showed VAT-excl…	summarize	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.74	28→36	–	$0.000027→$0.000249	yes
43	Translate to Swedish: 'Your payment could not be processed. Please check your c…	translate	gpt-4o-mini-2024-07-18	❌ 0.00	✅ 0.97	19→27	–	$0.000018→$0.00018	yes
44	Extract the tracking number: 'Your parcel 1Z999AA10123456784 left our facility …	extract	gpt-4o-mini-2024-07-18	✅ 1.00	✅ 1.00	8→12	–	$0.000012→$0.000111	yes
45	Extract the last four card digits: 'The charge went to the Visa ending in 4242.'	extract	gpt-4o-mini-2024-07-18	✅ 1.00	✅ 1.00	2→6	–	$0.000008→$0.000077	yes
46	Intent (question/complaint/praise/refund_request): 'Your support team solved my…	classify	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.10	1→25	–	$0.000008→$0.000174	yes
47	Extract as JSON {name, company}: 'Hi, this is Maria Gonzalez calling from Acme …	extract	gpt-4o-mini-2024-07-18	❌ 0.00	✅ 0.98	19→36	–	$0.00002→$0.000239	yes
48	Summarize this support thread in one sentence: 'Customer reported login failure…	summarize	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.38	20→34	–	$0.000024→$0.000248	yes
49	Priority (low/medium/high/urgent): 'A tooltip on the settings page has a small …	classify	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.13	3→44	–	$0.000009→$0.000269	yes
50	Make this subject line professional: 'wanna chat about the budget thing'	rewrite	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.43	7→18	–	$0.000011→$0.000131	yes
51	Extract as JSON {product, issue}: 'The X200 vacuum's battery drains within ten …	extract	gpt-4o-mini-2024-07-18	❌ 0.00	✅ 0.97	19→35	–	$0.000019→$0.000223	yes
52	Rewrite for a non-technical audience: 'The outage was caused by an expired TLS …	rewrite	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.50	15→24	–	$0.000017→$0.000174	yes
53	Translate to Polish: 'Two-factor authentication is now enabled on your account.'	translate	gpt-4o-mini-2024-07-18	❌ 0.00	✅ 0.91	24→29	–	$0.000021→$0.000188	yes
54	Extract the order number: 'Hello, I'm writing about order #84412-B which arrive…	extract	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.50	3→11	–	$0.000009→$0.000103	yes
55	Spam or not spam: 'URGENT: verify your account within 24 hours or it will be su…	classify	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.00	2→64	–	$0.000009→$0.00037	yes
56	Make this concise (max 10 words): 'Due to the fact that it was raining, we made…	rewrite	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.70	10→14	–	$0.000014→$0.000128	yes
57	Extract as JSON {street, zip, city}: 'Send returns to Lagerstrasse 12, 20095 Ha…	extract	gpt-4o-mini-2024-07-18	✅ 1.00	✅ 1.00	34→42	–	$0.000029→$0.000269	yes
58	Sentiment (positive/negative/neutral): 'It's fine. Does the job, nothing specia…	classify	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.06	2→47	–	$0.000008→$0.000282	yes
59	Headline (max 8 words) for this update: 'The mobile app now supports offline mo…	summarize	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.64	9→19	–	$0.000015→$0.000159	yes
60	Topic (billing/bug/feature_request/how_to): 'The dashboard shows a 500 error wh…	classify	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.42	20→42	–	$0.00002→$0.000264	yes
61	Rewrite in active voice: 'The proposal was reviewed by the committee.'	rewrite	gpt-4o-mini-2024-07-18	✅ 1.00	✅ 1.00	6→9	–	$0.00001→$0.000087	yes
62	Translate to Spanish: 'Click the link below to reset your password.'	translate	gpt-4o-mini-2024-07-18	❌ 0.00	✅ 0.99	15→24	–	$0.000015→$0.000161	yes
63	Does this text contain personal data? (yes/no): 'Bad weather in the region dela…	classify	gpt-4o-mini-2024-07-18	❌ 0.00	✅ 0.80	2→4	–	$0.000009→$0.000069	yes
64	Translate to Italian: 'The item is currently out of stock.'	translate	gpt-4o-mini-2024-07-18	❌ 0.00	✅ 0.97	10→14	–	$0.000012→$0.00011	yes
65	Extract all monetary amounts as a JSON array: 'Setup costs $500, then $99 per m…	extract	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.70	13→26	–	$0.000016→$0.000186	yes
66	Soften this for a performance review: 'Your code is sloppy and full of bugs.'	rewrite	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.44	14→33	–	$0.000015→$0.000212	yes
67	Translate to German: 'Your order has been shipped and will arrive in 3-5 busine…	translate	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.52	19→56	–	$0.000019→$0.000329	yes
68	Extract the version number: 'After updating to v2.14.3 the export button disapp…	extract	gpt-4o-mini-2024-07-18	❌ 0.00	✅ 0.92	5→9	–	$0.00001→$0.000092	yes
69	One-line summary of this changelog: 'Fixed a memory leak in the upload handler,…	summarize	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.47	19→22	–	$0.000021→$0.00017	yes
70	Extract the action items as a JSON array: 'John will email the report, Lisa boo…	extract	gpt-4o-mini-2024-07-18	❌ 0.00	✅ 0.90	66→91	–	$0.000048→$0.000511	yes
71	Translate to French: 'Your subscription will renew on July 1st.'	translate	gpt-4o-mini-2024-07-18	❌ 0.00	✅ 0.93	11→21	–	$0.000013→$0.000148	yes
72	TL;DR of these meeting notes in one sentence: 'We agreed to ship the beta on Ju…	summarize	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.63	34→42	–	$0.000032→$0.000287	yes
73	Summarize in one sentence for a status page: 'Between 09:12 and 10:47 UTC some …	summarize	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.46	39→35	–	$0.000035→$0.000255	yes
74	Route this support ticket (billing/technical/account/shipping): 'How do I downl…	classify	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.17	5→24	–	$0.000011→$0.000172	yes
75	Rewrite politely for a customer email: 'Send me the report now.'	rewrite	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.70	15→35	–	$0.000015→$0.000218	yes
76	Extract the username: 'User @data_wrangler_88 reported the sync issue first.'	extract	gpt-4o-mini-2024-07-18	❌ 0.00	✅ 0.97	6→11	–	$0.000011→$0.000102	yes
77	Extract the phone number: 'Call our support line at (415) 555-0132 between 9 an…	extract	gpt-4o-mini-2024-07-18	✅ 1.00	✅ 1.00	8→11	–	$0.000013→$0.000109	yes
78	Sentiment (positive/negative/neutral): 'Third time my delivery is late. I want …	classify	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.13	2→31	–	$0.000009→$0.000204	yes
79	Translate to Norwegian: 'Your return has been received and your refund is on it…	translate	gpt-4o-mini-2024-07-18	❌ 0.00	✅ 0.91	16→22	–	$0.000016→$0.000156	yes
80	Extract as JSON {day, time, timezone}: 'Let's sync Thursday at 14:30 CET on the…	extract	gpt-4o-mini-2024-07-18	✅ 1.00	✅ 1.00	30→38	–	$0.000026→$0.000244	yes
81	Make this more formal: 'Hey, can you take a look at this when you get a sec?'	rewrite	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.60	11→34	–	$0.000014→$0.000218	yes
82	Translate to Danish: 'Thank you for contacting support. We will reply within 24…	translate	gpt-4o-mini-2024-07-18	❌ 0.00	✅ 0.88	16→24	–	$0.000017→$0.000168	yes
83	Churn risk (low/medium/high): 'Renewed for another year, the new features are g…	classify	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.03	1→39	–	$0.000008→$0.000245	yes
84	Route this support ticket (billing/technical/account/shipping): 'I was charged …	classify	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.56	1→9	–	$0.000008→$0.000092	yes
85	Extract the license plate: 'The delivery van, plate HH-AB 1234, was parked at d…	extract	gpt-4o-mini-2024-07-18	✅ 1.00	✅ 1.00	6→10	–	$0.000012→$0.000103	yes
86	Extract as JSON {employee_id, department}: 'Badge E-20331 belongs to Priya Nair…	extract	gpt-4o-mini-2024-07-18	✅ 1.00	✅ 1.00	25→33	–	$0.000023→$0.00022	yes
87	Priority (low/medium/high/urgent): 'Login is broken for all users in production…	classify	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.06	4→123	–	$0.000009→$0.000661	yes
88	Summarize for a shift handover in one sentence: 'Night shift saw two failed bat…	summarize	gpt-4o-mini-2024-07-18	❌ 0.00	✅ 0.93	30→31	–	$0.000029→$0.000228	yes
89	Language of this text (ISO 639-1 code only): 'Hvor er nærmeste togstation?'	classify	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.00	1→4	–	$0.000008→$0.000071	yes
90	Intent (question/complaint/praise/refund_request): 'Why is my order still not h…	classify	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.19	2→23	–	$0.000009→$0.000167	yes
91	Extract the names of all people mentioned as a JSON array: 'Tom and Aisha met w…	extract	gpt-4o-mini-2024-07-18	✅ 1.00	✅ 1.00	16→23	–	$0.000018→$0.000171	yes
92	Moderation (allowed/flagged): 'You are an idiot and your product is garbage.'	classify	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.01	3→48	–	$0.000009→$0.000287	yes
93	Language of this text (ISO 639-1 code only): 'Wo ist der nächste Bahnhof?'	classify	gpt-4o-mini-2024-07-18	✅ 1.00	✅ 1.00	1→4	–	$0.000008→$0.000075	yes
94	Route this support ticket (billing/technical/account/shipping): 'My package sho…	classify	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.12	1→29	–	$0.000008→$0.000193	yes
95	Translate to Portuguese: 'Free shipping on orders over 50 euros.'	translate	gpt-4o-mini-2024-07-18	❌ 0.00	✅ 0.88	12→20	–	$0.000014→$0.000142	yes
96	Extract the email address: 'Reach me at jane.doe+work@example.co.uk anytime.'	extract	gpt-4o-mini-2024-07-18	✅ 1.00	✅ 1.00	9→14	–	$0.000012→$0.00012	yes
97	Summarize this bug report in one sentence: 'Steps: open the editor, paste a tab…	summarize	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.47	39→39	–	$0.000036→$0.000277	yes
98	Summarize the recurring feedback theme in one sentence: 'Across 80 survey respo…	summarize	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.58	22→21	–	$0.000023→$0.000173	yes
99	Extract the discount percentage: 'Use code SPRING for 15% off your first order.'	extract	gpt-4o-mini-2024-07-18	✅ 1.00	✅ 1.00	2→6	–	$0.000008→$0.000075	yes
100	Route this support ticket (billing/technical/account/shipping): 'The app crashe…	classify	gpt-4o-mini-2024-07-18	❌ 0.00	❌ 0.25	5→22	–	$0.00001→$0.000158	yes

Details

Failing rows first, then lowest mean score.

41. Churn risk (low/medium/high): 'We're evaluating alternatives because prices wen…

openai_gpt-4o-mini_69a66c3ebce30c3a → migration__openai_gpt-4o-mini_69a66c3ebce30c3a__to__claude-haiku-4-5 · semantic key c8437a70cc226f4269e81d6e5376ae31 · classify · out tokens 2→106

exact 0.00 — normalized texts differ
fuzzy 0.00 — sequence similarity 0.00 (threshold 0.8)

Prompt

[system] You are a precise assistant. Answer directly and concisely. No preamble.

[user] Churn risk (low/medium/high): 'We're evaluating alternatives because prices went up again.'

Baseline (gpt-4o-mini-2024-07-18)

Medium.

Target (claude-haiku-4-5-20251001)

**Churn risk: HIGH**

**Key indicators:**
- Explicit evaluation of competitors ("evaluating alternatives")
- Price sensitivity as primary driver
- Recurring frustration ("went up again" implies multiple increases)
- Active consideration stage, not just complaint

**Recommended actions:**
- Immediate outreach with retention offer
- Review pricing/contract terms for optimization
- Propose value-add alternatives to offset cost increase
- Escalate to retention specialist or account manager

55. Spam or not spam: 'URGENT: verify your account within 24 hours or it will be su…

openai_gpt-4o-mini_91e3187c78aa9a93 → migration__openai_gpt-4o-mini_91e3187c78aa9a93__to__claude-haiku-4-5 · semantic key 027bfa9a19cdee5b43841a6771753a21 · classify · out tokens 2→64

exact 0.00 — normalized texts differ
fuzzy 0.00 — sequence similarity 0.00 (threshold 0.8)

Prompt

[system] You are a precise assistant. Answer directly and concisely. No preamble.

[user] Spam or not spam: 'URGENT: verify your account within 24 hours or it will be suspended.'

Baseline (gpt-4o-mini-2024-07-18)

Spam.

Target (claude-haiku-4-5-20251001)

**Spam** (phishing attempt)

Characteristics:
- Artificial urgency ("URGENT", "24 hours")
- Threat of account suspension
- Requests verification (typical phishing tactic)
- Legitimate companies rarely use this approach via unsolicited messages

89. Language of this text (ISO 639-1 code only): 'Hvor er nærmeste togstation?'

openai_gpt-4o-mini_d9d019d6590c4ea8 → migration__openai_gpt-4o-mini_d9d019d6590c4ea8__to__claude-haiku-4-5 · semantic key 629a8cca6370942fa25d6a5b5b19c789 · classify · out tokens 1→4

exact 0.00 — normalized texts differ
fuzzy 0.00 — sequence similarity 0.00 (threshold 0.8)

Prompt

[system] You are a precise assistant. Answer directly and concisely. No preamble.

[user] Language of this text (ISO 639-1 code only): 'Hvor er nærmeste togstation?'

Baseline (gpt-4o-mini-2024-07-18)

da

Target (claude-haiku-4-5-20251001)

no

92. Moderation (allowed/flagged): 'You are an idiot and your product is garbage.'

openai_gpt-4o-mini_e0084440d3367cc4 → migration__openai_gpt-4o-mini_e0084440d3367cc4__to__claude-haiku-4-5 · semantic key ef72cd51a90ab8b41807dd733ad60e26 · classify · out tokens 3→48

exact 0.00 — normalized texts differ
fuzzy 0.01 — sequence similarity 0.01 (threshold 0.8)

Prompt

[system] You are a precise assistant. Answer directly and concisely. No preamble.

[user] Moderation (allowed/flagged): 'You are an idiot and your product is garbage.'

Baseline (gpt-4o-mini-2024-07-18)

Flagged.

Target (claude-haiku-4-5-20251001)

**Flagged**

This contains insults and harsh criticism that violates respectful communication standards. The use of "idiot" is a personal attack, and "garbage" is inflammatory language intended to demean.

7. Is this about a delivery problem? (yes/no): 'The tracking number you sent does …

openai_gpt-4o-mini_18e0f38c9da3089d → migration__openai_gpt-4o-mini_18e0f38c9da3089d__to__claude-haiku-4-5 · semantic key a49ec22ca5d985abc6a7bc862079e686 · classify · out tokens 2→43

exact 0.00 — normalized texts differ
fuzzy 0.02 — sequence similarity 0.02 (threshold 0.8)

Prompt

[system] You are a precise assistant. Answer directly and concisely. No preamble.

[user] Is this about a delivery problem? (yes/no): 'The tracking number you sent does not work.'

Baseline (gpt-4o-mini-2024-07-18)

Yes.

Target (claude-haiku-4-5-20251001)

No.

This is about a tracking issue, not a delivery problem. The customer is reporting that the tracking number itself is non-functional, rather than reporting that their delivery has encountered a problem.

83. Churn risk (low/medium/high): 'Renewed for another year, the new features are g…

openai_gpt-4o-mini_d08713e52b495d5f → migration__openai_gpt-4o-mini_d08713e52b495d5f__to__claude-haiku-4-5 · semantic key c073f1bc0de9ceee4df90d02e169d104 · classify · out tokens 1→39

exact 0.00 — normalized texts differ
fuzzy 0.03 — sequence similarity 0.03 (threshold 0.8)

Prompt

[system] You are a precise assistant. Answer directly and concisely. No preamble.

[user] Churn risk (low/medium/high): 'Renewed for another year, the new features are great.'

Baseline (gpt-4o-mini-2024-07-18)

Low

Target (claude-haiku-4-5-20251001)

**Low**

The statement indicates customer satisfaction with the product (praising new features) and commitment (renewed contract), both of which are strong indicators of low churn risk.

27. Spam or not spam: 'Congratulations! You won a $1000 gift card, click here to cl…

openai_gpt-4o-mini_43b5d4942e1dbb79 → migration__openai_gpt-4o-mini_43b5d4942e1dbb79__to__claude-haiku-4-5 · semantic key d06b7828215d1d402bb38e045ddc029b · classify · out tokens 2→54

exact 0.00 — normalized texts differ
fuzzy 0.05 — sequence similarity 0.05 (threshold 0.8)

Prompt

[system] You are a precise assistant. Answer directly and concisely. No preamble.

[user] Spam or not spam: 'Congratulations! You won a $1000 gift card, click here to claim.'

Baseline (gpt-4o-mini-2024-07-18)

Spam.

Target (claude-haiku-4-5-20251001)

**Spam**

Common spam indicators:
- Unsolicited congratulations/prize claim
- Urgency ("click here")
- Too-good-to-be-true offer
- Typical phishing/scam template

87. Priority (low/medium/high/urgent): 'Login is broken for all users in production…

openai_gpt-4o-mini_d80e914e1872be3d → migration__openai_gpt-4o-mini_d80e914e1872be3d__to__claude-haiku-4-5 · semantic key c84aa056273b4bbde705343b1d599ddd · classify · out tokens 4→123

exact 0.00 — normalized texts differ
fuzzy 0.06 — sequence similarity 0.06 (threshold 0.8)

Prompt

[system] You are a precise assistant. Answer directly and concisely. No preamble.

[user] Priority (low/medium/high/urgent): 'Login is broken for all users in production.'

Baseline (gpt-4o-mini-2024-07-18)

Priority: Urgent

Target (claude-haiku-4-5-20251001)

**Priority: URGENT**

This requires immediate action because:
- **Scope**: All users affected (not isolated)
- **Environment**: Production (live service)
- **Impact**: Complete service unavailability for authentication
- **Business risk**: Revenue loss, SLA violations, reputational damage

**Recommended immediate actions:**
1. Page on-call engineer/incident commander
2. Declare SEV-1 incident
3. Begin root cause investigation
4. Prepare status page communication
5. Activate war room if not already done

58. Sentiment (positive/negative/neutral): 'It's fine. Does the job, nothing specia…

openai_gpt-4o-mini_971ceef22d00b451 → migration__openai_gpt-4o-mini_971ceef22d00b451__to__claude-haiku-4-5 · semantic key 5bf36323c5f6371744833313a4747986 · classify · out tokens 2→47

exact 0.00 — normalized texts differ
fuzzy 0.06 — sequence similarity 0.06 (threshold 0.8)

Prompt

[system] You are a precise assistant. Answer directly and concisely. No preamble.

[user] Sentiment (positive/negative/neutral): 'It's fine. Does the job, nothing special.'

Baseline (gpt-4o-mini-2024-07-18)

Neutral.

Target (claude-haiku-4-5-20251001)

Neutral

The statement expresses acceptance and basic satisfaction without enthusiasm or criticism. The phrase "does the job" is functional/utilitarian, and "nothing special" explicitly indicates lack of strong positive or negative emotion.

32. Priority (low/medium/high/urgent): 'Checkout intermittently fails for about 10%…

openai_gpt-4o-mini_508d047cade5d1a6 → migration__openai_gpt-4o-mini_508d047cade5d1a6__to__claude-haiku-4-5 · semantic key b52db60497916ee283c01911dd51152a · classify · out tokens 3→86

exact 0.00 — normalized texts differ
fuzzy 0.08 — sequence similarity 0.08 (threshold 0.8)

Prompt

[system] You are a precise assistant. Answer directly and concisely. No preamble.

[user] Priority (low/medium/high/urgent): 'Checkout intermittently fails for about 10% of customers.'

Baseline (gpt-4o-mini-2024-07-18)

Priority: High

Target (claude-haiku-4-5-20251001)

**Priority: HIGH**

**Justification:**
- Direct revenue impact (10% of transactions failing)
- Customer-facing critical feature
- Affects purchasing capability
- High business cost despite "intermittent" nature

**Note:** Could escalate to URGENT if:
- Peak traffic period
- High-value customers affected
- Failure rate increasing
- Duration already extended

21. Route this support ticket (billing/technical/account/shipping): 'I can't reset …

openai_gpt-4o-mini_37d5392016c26bcf → migration__openai_gpt-4o-mini_37d5392016c26bcf__to__claude-haiku-4-5 · semantic key 014d3b6bef5d2295c99480cd61a3d3fa · classify · out tokens 1→6

exact 0.00 — normalized texts differ
fuzzy 0.10 — sequence similarity 0.10 (threshold 0.8)

Prompt

[system] You are a precise assistant. Answer directly and concisely. No preamble.

[user] Route this support ticket (billing/technical/account/shipping): 'I can't reset my password.'

Baseline (gpt-4o-mini-2024-07-18)

Technical

Target (claude-haiku-4-5-20251001)

**account**

46. Intent (question/complaint/praise/refund_request): 'Your support team solved my…

openai_gpt-4o-mini_719d4d1be2fdf365 → migration__openai_gpt-4o-mini_719d4d1be2fdf365__to__claude-haiku-4-5 · semantic key 95064c205282f4218605fc238b7a16c7 · classify · out tokens 1→25

exact 0.00 — normalized texts differ
fuzzy 0.10 — sequence similarity 0.10 (threshold 0.8)

Prompt

[system] You are a precise assistant. Answer directly and concisely. No preamble.

[user] Intent (question/complaint/praise/refund_request): 'Your support team solved my issue in minutes!'

Baseline (gpt-4o-mini-2024-07-18)

Praise

Target (claude-haiku-4-5-20251001)

**Intent: Praise**

The customer is expressing satisfaction with the support team's quick and effective resolution.

9. Topic (billing/bug/feature_request/how_to): 'How do I add a teammate to my work…

openai_gpt-4o-mini_1b7f67df4a19f8dc → migration__openai_gpt-4o-mini_1b7f67df4a19f8dc__to__claude-haiku-4-5 · semantic key cdae929542ed548a8e57aca83344fb80 · classify · out tokens 48→198

exact 0.00 — normalized texts differ
fuzzy 0.12 — sequence similarity 0.12 (threshold 0.8)

Prompt

[system] You are a precise assistant. Answer directly and concisely. No preamble.

[user] Topic (billing/bug/feature_request/how_to): 'How do I add a teammate to my workspace?'

Baseline (gpt-4o-mini-2024-07-18)

To add a teammate to your workspace, go to the workspace settings, select "Members," and then click on "Invite Team Member" or "Add Member." Enter the email address of the teammate you want to invite and send the invitation.

Target (claude-haiku-4-5-20251001)

# How to Add a Teammate to Your Workspace

1. **Go to Workspace Settings**
   - Click on your workspace name in the top left
   - Select "Settings" or "Members"

2. **Invite Team Members**
   - Click "Add Member" or "Invite"
   - Enter their email address
   - Select their role/permission level

3. **Send Invitation**
   - Click "Send Invite" or "Confirm"
   - They'll receive an email invitation
   - They can accept and join your workspace

4. **Set Permissions** (optional)
   - Assign appropriate access level (Admin, Editor, Viewer, etc.)
   - Configure which projects/resources they can access

**Note:** The exact steps vary by platform. If you need help with a specific tool, provide the workspace name and I can give more detailed instructions.

94. Route this support ticket (billing/technical/account/shipping): 'My package sho…

openai_gpt-4o-mini_ee8e8bb716aaf7ba → migration__openai_gpt-4o-mini_ee8e8bb716aaf7ba__to__claude-haiku-4-5 · semantic key fb1e148017a6803bf11d3d0a11582df0 · classify · out tokens 1→29

exact 0.00 — normalized texts differ
fuzzy 0.12 — sequence similarity 0.12 (threshold 0.8)

Prompt

[system] You are a precise assistant. Answer directly and concisely. No preamble.

[user] Route this support ticket (billing/technical/account/shipping): 'My package shows delivered but never arrived.'

Baseline (gpt-4o-mini-2024-07-18)

Shipping

Target (claude-haiku-4-5-20251001)

**Route: Shipping**

This is a delivery/fulfillment issue that requires investigation by the shipping/logistics department.

78. Sentiment (positive/negative/neutral): 'Third time my delivery is late. I want …

openai_gpt-4o-mini_c6f34a2dbb588911 → migration__openai_gpt-4o-mini_c6f34a2dbb588911__to__claude-haiku-4-5 · semantic key b34a1e8841bcfaa4226ea10d738a5154 · classify · out tokens 2→31

exact 0.00 — normalized texts differ
fuzzy 0.13 — sequence similarity 0.13 (threshold 0.8)

Prompt

[system] You are a precise assistant. Answer directly and concisely. No preamble.

[user] Sentiment (positive/negative/neutral): 'Third time my delivery is late. I want a refund.'

Baseline (gpt-4o-mini-2024-07-18)

Negative.

Target (claude-haiku-4-5-20251001)

**Negative**

The customer expresses frustration over repeated late deliveries and is requesting a refund, indicating dissatisfaction.

5. Sentiment (positive/negative/neutral): 'Shipping was fast but the product broke…

openai_gpt-4o-mini_0d22317379c5e062 → migration__openai_gpt-4o-mini_0d22317379c5e062__to__claude-haiku-4-5 · semantic key 03be769ed6dcff44db3d26045168f978 · classify · out tokens 2→31

exact 0.00 — normalized texts differ
fuzzy 0.13 — sequence similarity 0.13 (threshold 0.8)

Prompt

[system] You are a precise assistant. Answer directly and concisely. No preamble.

[user] Sentiment (positive/negative/neutral): 'Shipping was fast but the product broke within a week.'

Baseline (gpt-4o-mini-2024-07-18)

Negative.

Target (claude-haiku-4-5-20251001)

**Negative**

The negative aspect (product breaking) outweighs the positive (fast shipping), resulting in overall negative sentiment.

49. Priority (low/medium/high/urgent): 'A tooltip on the settings page has a small …

openai_gpt-4o-mini_7dff935818428c9d → migration__openai_gpt-4o-mini_7dff935818428c9d__to__claude-haiku-4-5 · semantic key 7c6853ae24d2c12058dc8aca810a418d · classify · out tokens 3→44

exact 0.00 — normalized texts differ
fuzzy 0.13 — sequence similarity 0.13 (threshold 0.8)

Prompt

[system] You are a precise assistant. Answer directly and concisely. No preamble.

[user] Priority (low/medium/high/urgent): 'A tooltip on the settings page has a small typo.'

Baseline (gpt-4o-mini-2024-07-18)

Priority: Low

Target (claude-haiku-4-5-20251001)

**Priority: Low**

A tooltip typo is a minor cosmetic issue with no impact on functionality, user safety, or core experience. It can be addressed in a routine update or maintenance cycle.

22. Spam or not spam: 'Hi Sam, can we move our 2pm meeting to 3pm tomorrow?'

openai_gpt-4o-mini_39c0d06de1f8b940 → migration__openai_gpt-4o-mini_39c0d06de1f8b940__to__claude-haiku-4-5 · semantic key c8064709cfc5372f50f9983ffe382dc5 · classify · out tokens 3→23

exact 0.00 — normalized texts differ
fuzzy 0.16 — sequence similarity 0.16 (threshold 0.8)

Prompt

[system] You are a precise assistant. Answer directly and concisely. No preamble.

[user] Spam or not spam: 'Hi Sam, can we move our 2pm meeting to 3pm tomorrow?'

Baseline (gpt-4o-mini-2024-07-18)

Not spam.

Target (claude-haiku-4-5-20251001)

Not spam.

This is a legitimate personal or professional communication requesting to reschedule a meeting.

74. Route this support ticket (billing/technical/account/shipping): 'How do I downl…

openai_gpt-4o-mini_bf4cf18052faa0ad → migration__openai_gpt-4o-mini_bf4cf18052faa0ad__to__claude-haiku-4-5 · semantic key 4ccf03628906ff9b4509f832b641196e · classify · out tokens 5→24

exact 0.00 — normalized texts differ
fuzzy 0.17 — sequence similarity 0.17 (threshold 0.8)

Prompt

[system] You are a precise assistant. Answer directly and concisely. No preamble.

[user] Route this support ticket (billing/technical/account/shipping): 'How do I download my invoices for tax season?'

Baseline (gpt-4o-mini-2024-07-18)

Route to: billing.

Target (claude-haiku-4-5-20251001)

**Account**

This request relates to accessing account documents and financial records, which falls under account management.

90. Intent (question/complaint/praise/refund_request): 'Why is my order still not h…

openai_gpt-4o-mini_dab2ba3fba1c2c28 → migration__openai_gpt-4o-mini_dab2ba3fba1c2c28__to__claude-haiku-4-5 · semantic key a9852fe03e6143801baea8c1d1cd5cd8 · classify · out tokens 2→23

exact 0.00 — normalized texts differ
fuzzy 0.19 — sequence similarity 0.19 (threshold 0.8)

Prompt

[system] You are a precise assistant. Answer directly and concisely. No preamble.

[user] Intent (question/complaint/praise/refund_request): 'Why is my order still not here after two weeks?'

Baseline (gpt-4o-mini-2024-07-18)

Complaint.

Target (claude-haiku-4-5-20251001)

**Intent: Complaint**

The customer is expressing dissatisfaction about a delayed order delivery.

23. Intent (question/complaint/praise/refund_request): 'The blender stopped working…

openai_gpt-4o-mini_3d25d5ef9bc6a08a → migration__openai_gpt-4o-mini_3d25d5ef9bc6a08a__to__claude-haiku-4-5 · semantic key 567604ac5581ce8758b2be9a66bf756d · classify · out tokens 3→37

exact 0.00 — normalized texts differ
fuzzy 0.19 — sequence similarity 0.19 (threshold 0.8)

Prompt

[system] You are a precise assistant. Answer directly and concisely. No preamble.

[user] Intent (question/complaint/praise/refund_request): 'The blender stopped working after two uses. I want my money back.'

Baseline (gpt-4o-mini-2024-07-18)

Refund request.

Target (claude-haiku-4-5-20251001)

**Intent: Refund Request**

**Reason:** Customer is requesting a refund due to a defective product (blender stopped working after minimal use).

26. Topic (billing/bug/feature_request/how_to): 'It would be great if exports suppo…

openai_gpt-4o-mini_43a46c2898a7ac58 → migration__openai_gpt-4o-mini_43a46c2898a7ac58__to__claude-haiku-4-5 · semantic key 2ba2e95c8a199d9e9d4582cb0f381f4b · classify · out tokens 3→29

exact 0.00 — normalized texts differ
fuzzy 0.24 — sequence similarity 0.24 (threshold 0.8)

Prompt

[system] You are a precise assistant. Answer directly and concisely. No preamble.

[user] Topic (billing/bug/feature_request/how_to): 'It would be great if exports supported CSV.'

Baseline (gpt-4o-mini-2024-07-18)

Feature Request.

Target (claude-haiku-4-5-20251001)

**Topic: feature_request**

This is a request for new functionality (CSV export support) that doesn't currently exist.

100. Route this support ticket (billing/technical/account/shipping): 'The app crashe…

openai_gpt-4o-mini_f7fe0e021a97d678 → migration__openai_gpt-4o-mini_f7fe0e021a97d678__to__claude-haiku-4-5 · semantic key ed3788822c7af6968f88bcb94a504b0a · classify · out tokens 5→22

exact 0.00 — normalized texts differ
fuzzy 0.25 — sequence similarity 0.25 (threshold 0.8)

Prompt

[system] You are a precise assistant. Answer directly and concisely. No preamble.

[user] Route this support ticket (billing/technical/account/shipping): 'The app crashes when I open settings.'

Baseline (gpt-4o-mini-2024-07-18)

Route to technical support.

Target (claude-haiku-4-5-20251001)

**Technical**

This is a software functionality issue requiring troubleshooting and bug investigation.

48. Summarize this support thread in one sentence: 'Customer reported login failure…

openai_gpt-4o-mini_783e9d41ed03ffc2 → migration__openai_gpt-4o-mini_783e9d41ed03ffc2__to__claude-haiku-4-5 · semantic key a3fd53885f3b2ddb7055dce2de11f611 · summarize · out tokens 20→34

exact 0.00 — normalized texts differ
fuzzy 0.38 — sequence similarity 0.38 (threshold 0.8)

Prompt

[system] You are a precise assistant. Answer directly and concisely. No preamble.

[user] Summarize this support thread in one sentence: 'Customer reported login failures on mobile, support suggested clearing the cache which did not help, the issue was escalated to engineering who found an expired certificate, a fix was deployed, and the customer confirmed resolution.'

Baseline (gpt-4o-mini-2024-07-18)

A customer experienced login failures on mobile, which were resolved after engineering identified and fixed an expired certificate.

Target (claude-haiku-4-5-20251001)

A customer's mobile login failures caused by an expired certificate were resolved after support escalated to engineering, deployed a fix, and confirmed the issue was fixed.

60. Topic (billing/bug/feature_request/how_to): 'The dashboard shows a 500 error wh…

openai_gpt-4o-mini_9ad3271e2ca360d8 → migration__openai_gpt-4o-mini_9ad3271e2ca360d8__to__claude-haiku-4-5 · semantic key e2ab3a1b05ca5834dc13f15cf52ed565 · classify · out tokens 20→42

exact 0.00 — normalized texts differ
fuzzy 0.42 — sequence similarity 0.42 (threshold 0.8)

Prompt

[system] You are a precise assistant. Answer directly and concisely. No preamble.

[user] Topic (billing/bug/feature_request/how_to): 'The dashboard shows a 500 error when filtering by date.'

Baseline (gpt-4o-mini-2024-07-18)

Topic: Bug  
Description: The dashboard displays a 500 error when attempting to filter by date.

Target (claude-haiku-4-5-20251001)

**Topic: Bug**

This is a technical malfunction where the dashboard fails to process a date filter request, resulting in a server error (500). It should be reported as a bug.

… 75 more rows omitted; see the Results table or pass --max-detail-rows 0.

Pricing

Cost is derived at report time from recorded tokens and the snapshots below; cassettes store tokens only. * marks estimates where some token categories had no rate in the profile.

anthropic-list+openai-list (USD, as-of latest): anthropic-list/2026-06-12.json (effective 2026-06-12, sha256 8197d3770986…); openai-list/2026-06-12.json (effective 2026-06-12, sha256 f850297a7d4f…)