Core Concepts

Three ideas explain why Distil's numbers are real and why the approach is different: decision-equivalence as the correct quality target, a statistical certificate that enforces it, and cache-aware cost modeling that makes aggressive compression financially correct.

Interactive — click any block to open the concept or technique behind it.

1. Decision-equivalence, not byte-equivalence

Most context compressors frame the problem as a string transformation: shrink the context, compare the output byte-by-byte, report overlap as accuracy. That framing has a fatal flaw — byte-equivalence and high compression are information-theoretically in tension. A claim of "87% compression, 100% accuracy" under byte-equivalence is self-contradicting; the two numbers cannot describe the same operation.

Decision-equivalence is the correct target. An agent only needs to take the same actions and produce the same outputs on the compressed context as it would on the original. "Same decision" means:

• The same tool calls (name + arguments) in the same order.
• The same final answer or action at each turn.
• Measured across a real multi-turn trajectory, not a single prompt.

This is measurable, certifiable, and compatible with aggressive compression. Context blocks that are causally inert — they never change a decision when removed — are provably free to drop, regardless of how many bytes they contain. Distil's salience protection identifies the load-bearing frontier (model-free, using a causal ablation pass rather than a scorer model) and ensures those blocks are never touched, regardless of compression depth.

Two recently shipped techniques apply decision-equivalence directly to the coding-agent read → edit → re-read loop. Cache-delta context coding (distil/cachedelta.py) replaces an exact or near-duplicate re-read with a back-reference or reference + diff. It is decision-equivalent because the prior version of the file is already present earlier in the cached conversation — prior version + diff carries the same information. It also enforces cache-monotonicity: only the volatile suffix is mutated, so every prompt-cache hit that existed before the turn still exists after it. The deepest layer, AST-structural delta (distil/astdelta.py), fingerprints Python definitions with ast.dump (invariant to whitespace, comments, and import order), so a reformat-only re-read is a zero-change reference rather than a spurious diff. For a full treatment see Techniques → Cache-delta and AST-structural delta.

2. The quality contract

A compression strategy in Distil does not ship unless a pre-registered statistical gate certifies it. Distil uses two complementary certification methods, both grounded in conformal prediction:

• TOST (Two One-Sided Tests) — the standard non-inferiority framework used in pharmaceutical trials. Pre-register a margin (default: 2 pp drop in task-success rate) and significance level (default: α = 0.05). If the lower one-sided test rejects H₀ that mean ≤ −margin, the strategy is certified non-inferior.
• DERC (Distribution-free Equivalence with Risk Control) — a Learn-Then-Test / CRC certificate that gives P(R(λ̂) ≤ α) ≥ 1−δ with Hoeffding–Bentkus p-values. No distributional assumptions; valid at finite sample sizes.

The Student-t tail used in TOST is computed from a hand-rolled regularised incomplete beta function — zero dependencies on scipy or numpy. Both gates run across all 7 corpus domains.

How the TOST gate works

1. Pre-register the indifference margin and significance level.
2. For each turn, run the agent on the original context and on the compressed context; record whether the decision matched.
3. Compute paired differences (compressed score − baseline score) across all turns.
4. Run the lower one-sided test. Reject H₀ at α → the strategy is certified non-inferior.

$ distil certify --strategy distil
decision-equivalence match rate: 100.0%
TOST non-inferiority (margin=0.02, alpha=0.05): mean diff=+0.000, p=0.0000

VERDICT: PASS  (certified non-inferior)

$ distil certify --strategy aggressive
decision-equivalence match rate: 0.0%
TOST non-inferiority (margin=0.02, alpha=0.05): mean diff=−1.000, p=1.0000

VERDICT: FAIL  (NOT certified — would degrade quality)

A strategy that cannot pass the gate simply does not ship. You can watch the gate reject a quality-degrading strategy yourself — it is not a configuration option.

3. Cache misses — not size — dominate agent cost

This is the insight most context compressors miss entirely, and it is where Distil's biggest wins come from.

In a multi-turn agent loop, the growing context is re-sent to the model at every turn. With prompt caching enabled:

• A cache read costs roughly 0.1× the normal input price.
• A cache write costs roughly 1.25× of normal input.
• A cache miss (fresh token) costs the full input price.

The dominant cost lever is therefore not how many tokens you send — it is how many of those tokens hit the cache. A naive compressor that rewrites the context every turn drops the longest common prefix to zero, losing the 10× discount on every previously-cached block.

Distil's cache-aware engine models this explicitly. It keeps the prefix byte-stable across turns via schema canonicalization and volatile-field extraction, compresses only the volatile tail, and proves via simulation that the naive approach is strictly dominated.

The numbers, measured

The 33% figure is the conservative lossless, single-trajectory result. Across a varied 64-trajectory corpus, the certified lossless ceiling is ~58%. The distil-causal mode reaches ~81% on that offline corpus. And it holds up where it counts — under live model grading: in the live head-to-head against the real installed packages (a realistic, decision-determined corpus, graded by claude-opus-4-8), Distil certified 83.2% savings at a 0% decision-change rate, while LLMLingua-2 reached only 53.1% savings and flipped 1-in-5 decisions, and Headroom achieved 35.3% at 0%.

For the compression mechanics, see Techniques. To price any trajectory yourself, see CLI Reference → distil savings.