Repairing the quoted values in written context, 2026-09-13
gemini-2.5-flash, 3 repetitions, gate-cleaned golds, 300s timeout, 100k row cap.

without-context arm = the question's own evidence sentence, as written.
with-context arm    = the same sentence with every quoted value repaired
                      against the real column values in the database.
So a POSITIVE lift means the repair helped.

What the repair changed, from scripts/repair_literals.py:
  toxicology             26 literals in 23 sentences   ' = '        -> '='
  card_games             14 literals in 12 sentences   'restricted' -> 'Restricted'
                                                        'legal'     -> 'Legal'
  thrombosis_prediction   0 literals; 45 left alone, mostly date and
                          numeric columns a text-value scan cannot see
Nothing was guessed: a literal is replaced only when exactly one real
column value matches it after trimming and case-folding.

=================== toxicology ===================
pooled 3 repetitions per arm

paired ablation on 417 observations (139 questions x 3 reps, execution accuracy)
  without context :  44.4%  [39.7%, 49.2%]
  with context    :  45.3%  [40.6%, 50.1%]
  lift            :  +1.0%  [-2.5%, +6.8%]   (the two rates subtracted; 95% interval on the paired difference, per question)

  each question counted once - the number to defend
    fixed 7 / broke 4 of 139 questions
    p = 0.5488   not significant at 0.05

  each answer counted separately - optimistic, because repeated
  answers to one question are not independent
    fixed 25 / broke 21 of 417 observations

=================== thrombosis_prediction ===================
pooled 3 repetitions per arm

paired ablation on 429 observations (143 questions x 3 reps, execution accuracy)
  without context :  41.0%  [36.5%, 45.7%]
  with context    :  40.1%  [35.6%, 44.8%]
  lift            :  -0.9%  [-3.8%, +2.4%]   (the two rates subtracted; 95% interval on the paired difference, per question)

  each question counted once - the number to defend
    fixed 2 / broke 3 of 143 questions
    p = 1.0000   not significant at 0.05

  each answer counted separately - optimistic, because repeated
  answers to one question are not independent
    fixed 14 / broke 18 of 429 observations

=================== card_games ===================
pooled 3 repetitions per arm

paired ablation on 504 observations (168 questions x 3 reps, execution accuracy)
  without context :  39.1%  [34.9%, 43.4%]
  with context    :  40.7%  [36.5%, 45.0%]
  lift            :  +1.6%  [-0.9%, +5.7%]   (the two rates subtracted; 95% interval on the paired difference, per question)

  each question counted once - the number to defend
    fixed 6 / broke 2 of 168 questions
    p = 0.2891   not significant at 0.05

  each answer counted separately - optimistic, because repeated
  answers to one question are not independent
    fixed 15 / broke 7 of 504 observations

=================== POOLED, 3 databases ===================
pooled 3 repetitions per arm

paired ablation on 1350 observations (450 questions x 3 reps, execution accuracy)
  without context :  41.3%  [38.7%, 44.0%]
  with context    :  41.9%  [39.3%, 44.6%]
  lift            :  +0.6%  [-0.8%, +3.5%]   (the two rates subtracted; 95% interval on the paired difference, per question)

  each question counted once - the number to defend
    fixed 15 / broke 9 of 450 questions
    p = 0.3075   not significant at 0.05

  each answer counted separately - optimistic, because repeated
  answers to one question are not independent
    fixed 54 / broke 46 of 1350 observations
    p = 0.4841   not significant at 0.05
  answers changed at 641 of 1350 pairs; 100 changed the verdict, 541 moved without changing it

=================== what the arm cost ===================
spent: 417 calls, 171,315 prompt + 28,714 output + 86,738 thinking tokens, $0.3400
spent: 429 calls, 331,989 prompt + 30,921 output + 96,150 thinking tokens, $0.4173
spent: 504 calls, 605,715 prompt + 23,144 output + 80,125 thinking tokens, $0.4399
