Architecture Overview

How the parts fit together: the tier model, the measure→discover→certify loop, byte-fidelity invariants, auth-mode gating, and holdout A/B validation.

The tier model

Distil organises every compression operation into three tiers, ordered by safety. Higher tiers require stronger evidence before they are permitted.

The measure → discover → certify loop

This is the architecture that makes the quality contract meaningful — and that separates Distil from estimators that measure quality and savings in separate runs.

Step 1 — Measure

For each trajectory and strategy, the cache-aware cost engine (compress/cache_aware.py) simulates the full multi-turn loop with four variants: baseline (no cache, no compress), cache only, naive compress + cache, and distil. It computes per-turn cost breakdowns and reports the total dollar cost of each strategy.

Step 2 — Discover

The causal ablation engine (replay/ablation.py) systematically removes each context block, replays the trajectory through the runner, and records whether any decision changed. The output is a per-block verdict: causally inert (safe to prune) or decision-driving (must be kept). This step produces the compression policy — it does not assume one.

Step 3 — Certify

The TOST gate (certify/stats.py via certify/gate.py) computes paired differences (compressed score − baseline score) across all turns and runs a one-sided Student-t test against the pre-registered margin. The implementation uses a hand-rolled regularised incomplete beta function — zero scipy/numpy.

A non-inferior strategy has its policy registered in the compress registry. A failing strategy is rejected and produces a non-zero exit code from distil certify and distil bench.

Byte-fidelity invariants

Module: distil/fidelity.py · Gate: distil verify

Two structural invariants are enforced across the entire corpus by the verify gate:

Reversibility

Every original block is recoverable, either because the compressed text is byte-identical to the original, or because the original lives in the local restore table (keyed by block ID for Tier-0, by content handle for Tier-1). Recovery is confirmed via SHA-256 equality — auditable and machine-checkable.

Append-only history

A block ID that appears in two consecutive turns must carry identical bytes. Mutating a previously-seen block ID is a violation. This invariant ensures the prefix cache is never invalidated by history rewriting — the cache hit rate depends on it.

Numeric precision

JSON canonicalization must not lose numeric precision. numeric_precision_preserved() confirms that both the original and transformed strings parse to the same JSON value.

$ distil verify
byte-fidelity gate — reversibility + append-only across the corpus

  sre-disk-incident        reversible + append-only: ok
  coding-bugfix            reversible + append-only: ok
  support-refund           reversible + append-only: ok
  research-synthesis       reversible + append-only: ok
  data-analysis-sql        reversible + append-only: ok
  devops-rollback          reversible + append-only: ok
  finance-reconcile        reversible + append-only: ok

FIDELITY: PASS — Tier-0/1 byte-reversible and history append-only across the corpus.

Auth-mode gating

Module: distil/policy.py

Aggressive compression applied to a subscription or OAuth session can alter conversations in ways that violate provider terms — injected retrieval tools the user never authorized, rewritten history that changes the conversation record. Auth-mode gating is a safety boundary, not an optimization.

The policy is a tightening boundary — a project's config can never loosen it. A PolicyError is raised if a non-permitted strategy is requested under the active auth mode. This maps cleanly onto the proxy's --lossless-only flag for subscription-context deployments.

Holdout A/B with bootstrap CI

Module: distil/certify/holdout.py · Command: distil holdout

The holdout module provides an additional validation layer on top of the TOST gate. It:

1. Deterministically partitions the corpus into a control group (default: 20%, held out and not counted toward the headline) and a treatment group, using a hash-based split for reproducibility.
2. Measures savings in both groups separately.
3. Bootstrap-resamples the treatment group to compute a 95% confidence interval on the savings estimate.

The reported confidence interval is honest: it reflects the uncertainty in the savings measurement given the finite corpus size. The control group acts as a sanity check — if control and treatment savings diverge significantly, something is wrong with the partition or the measurement.

Module map