clef-compactor — keep the evidence, cut the noise

how it works

One question per chunk. One call per 64 chunks.

Clef answers P(relevant) for every chunk against your query. The compactor ranks by score, drops what sits below the threshold, and fills the remaining budget with the best chunks. Every cut carries a reason your code can log or display.

pipeline

The gate, animated

Batched noul questions, deterministic ranking, greedy budget fill.

verbatim

Never rewritten

Kept chunks keep their exact text. Summaries would break citations; deletion keeps them honest.

auditable

Reasoned cuts

Each dropped chunk records irrelevant or budget_exhausted, plus its score and token count.

measured results

Real model, measured on real hardware

Open-weights clef-flash 9B, float16 on 2x Kaggle T4, 20 hand-labelled cases, 104 chunks. Reproducible from the public kernel. The replay baseline (simulated scorer) and a hosted-API run are documented in the repo.

metric	open-weights 9B on 2xT4	replay baseline
chunk accuracy	0.712	0.990
kept precision	0.771	1.000
relevant recall	0.746	0.984
kept F1	0.758	0.992
context tokens saved	38.4%	34.0%
scoring latency p50	1,274 ms	<1 ms
cost per 1k calls	$0.00 self-hosted	$0.019

clef vs laya

Different points on the same trade-off

Published numbers from Cloudflare's "Introducing Clef" blog post. Laya runs locally and wins raw latency; Clef wins decision quality by a wide margin; clef-flash sits in between.

benchmark	clef	clef-flash	laya
BFCL, case exact	98.47	98.76	38.13
ToolRet, nDCG@10	69.19	66.43	12.69
API-Bank accuracy	91.93	93.11	11.41
median latency, ms	209.3	38.8	5.8
p95 latency, ms	238.6	122.4	222.5
context window	65,536	65,536	32k reported

quick start

Three lines to a smaller context

from clef_compactor import ClefCompactor

compactor = ClefCompactor()  # reads CLEF_ACCOUNT_ID / CLEF_API_TOKEN
result = compactor.compact("What is the refund policy?", chunks, token_budget=1000)
context = "\n\n".join(result.kept_texts())

Async client, CLI, an OpenAI-compatible endpoint and LangChain / LlamaIndex adapters ship in the box. See the README for copy-paste examples of each.

limitations

What this tool is not

Not a quality certificateThe committed accuracy comes from a simulated scorer. It proves the pipeline ranks and cuts correctly; run --mode live before quoting model quality.
Not the fastest optionClef's median decision latency is 209 ms plus network. Laya stays local at 5.8 ms. If you need sub-10 ms on every request, use laya-compactor.
Not exact token mathBudgets are enforced with the cl100k_base encoding, a close proxy for Cloudflare's tokenizer rather than the tokenizer itself.
Not graded relevanceClef answers P(relevant). There is no supporting-versus-essential signal, and the 0.5 threshold is a blunt but predictable cut.