We froze everything — the store (rebuilt byte-identically from its event log), the task suite (SHA-pinned before any measurement), the prompts, the temperature — and changed exactly one thing: the model. Qwen2.5 at 7B, 14B, 32B, and 72B, each driving the same operator toolbox and the same retrieval baselines, on H100 and Ada GPUs.
The interface is the multiplier
This is the cleanest evidence yet for the claim that started this project: the interface is what converts model capability into answer quality. A stronger planner writes better operator DAGs, so every increment of model quality flows straight into accuracy — 0.138 at 7B, 0.340 at 14B, 0.628 at 32B. Meanwhile the same stronger models gained the baselines almost nothing, because no amount of reasoning fixes an input that doesn't contain the answer's structure: serialized edges still don't compose, and generated Cypher still can't express belief-state time travel.
The belief-state result is the sharpest version: correction probes — "as of transaction time T, what did we believe?" — went 0.38 → 0.77 → 1.000. A perfect score at 32B means the bi-temporal machinery isn't aspirational headroom; a good enough planner uses it flawlessly, today, with open weights on one GPU.
Surprise #1: the 72B model got worse
The biggest model in the sweep — 72B, five times the parameters of our 32B point — scored lower: 0.511 overall, and a striking 0.31 on probes. The asterisk explains it: 72B only fits a single 80 GB GPU with AWQ 4-bit quantization, and every other point in the curve ran full-precision. Quantization, not scale, is the planning bottleneck: the compressed model writes visibly worse structured plans even though its general benchmark scores barely move. That matters for deployment, because AWQ-class quantization is exactly what small-GPU serving uses. If your agent plans against tools, benchmark your quantized model — the degradation concentrates precisely in structured planning.
Surprise #2: we un-hobbled RAG, and it got worse
Since our first writeup, the fairest criticism has been: "your vector-RAG baseline only retrieved one chunk because your GPU was small." True — a 24 GB card's serving window forced k=1. So on an H100, at the model's full native 32k context, we ran the intended k=20 retrieval breadth. The result:
| vector-RAG config | corpus exposure | tokens/task | exact match |
|---|---|---|---|
| k=1 × 256-event chunks (24 GB card) | ~0.4% | ~19k | 0.106 |
| k=20 × 24-event chunks (H100, 32k) | ~0.8% × 20 sites | 32,475 | 0.021 |
| TGMS, same run, same model | — | 6,521 | 0.362 |
More context made it worse — 17× behind TGMS at 5× the token cost. Retrieval breadth was never the bottleneck; representation was. Spreading attention across twenty chunks of serialized edges dilutes exactly the signal the questions quantify over. You cannot prompt your way to a temporal join.
The number that never moved
Across every cell of this study — four scales, two quantizations, plus two other model families (Llama-3.1-8B and Phi-4-mini) driving the same toolbox — the emitted unsupported-claim rate was 0.000. Weaker models answered fewer questions; they never answered more of them wrongly, because claims that don't verify against the execution trace never ship. That's the architecture's contract: competence lives outside the trust boundary and scales with the model; safety lives inside it and doesn't have to.
tgms replay), on the
University of Memphis iTiger GPU cluster; receipts and per-class tables
in the
technical report §8.2c and the
v0.3.0
release. The scale figure appears in the updated
paper.