clef-compactor — keep the evidence, cut the noise

how it works

One question per chunk. One call per 64 chunks.

Clef answers P(relevant) for every chunk against your query. The compactor ranks by score, drops what sits below the threshold, and fills the remaining budget with the best chunks. Every cut carries a reason your code can log or display.

pipeline

The gate, animated

Batched noul questions, deterministic ranking, greedy budget fill.

verbatim

Never rewritten

Kept chunks keep their exact text. Summaries would break citations; deletion keeps them honest.

auditable

Reasoned cuts

Each dropped chunk records irrelevant or budget_exhausted, plus its score and token count.

measured results

Replay baseline, committed and reproducible

Deterministic simulated scorer, 20 cases, 104 chunks, seed 20261001. This validates the pipeline; it does not measure Clef. Live numbers land via evals/run_eval.py --mode live.

metric	value
chunk accuracy	0.990
kept precision	1.000
relevant recall	0.984
kept F1	0.992
context tokens saved	34.0%
cost per 1k calls	$0.019

clef vs laya

Different points on the same trade-off

Published numbers from Cloudflare's "Introducing Clef" blog post. Laya runs locally and wins raw latency; Clef wins decision quality by a wide margin; clef-flash sits in between.

benchmark	clef	clef-flash	laya
BFCL, case exact	98.47	98.76	38.13
ToolRet, nDCG@10	69.19	66.43	12.69
API-Bank accuracy	91.93	93.11	11.41
median latency, ms	209.3	38.8	5.8
p95 latency, ms	238.6	122.4	222.5
context window	65,536	65,536	32k reported

quick start

Three lines to a smaller context

from clef_compactor import ClefCompactor

compactor = ClefCompactor()  # reads CLEF_ACCOUNT_ID / CLEF_API_TOKEN
result = compactor.compact("What is the refund policy?", chunks, token_budget=1000)
context = "\n\n".join(result.kept_texts())

Async client, CLI, an OpenAI-compatible endpoint and LangChain / LlamaIndex adapters ship in the box. See the README for copy-paste examples of each.

limitations

What this tool is not

Not a quality certificateThe committed accuracy comes from a simulated scorer. It proves the pipeline ranks and cuts correctly; run --mode live before quoting model quality.
Not the fastest optionClef's median decision latency is 209 ms plus network. Laya stays local at 5.8 ms. If you need sub-10 ms on every request, use laya-compactor.
Not exact token mathBudgets are enforced with the cl100k_base encoding, a close proxy for Cloudflare's tokenizer rather than the tokenizer itself.
Not graded relevanceClef answers P(relevant). There is no supporting-versus-essential signal, and the 0.5 threshold is a blunt but predictable cut.