contexel — deterministic context shaping for code-writing agents

Deterministic, dependency-free context shaping for code-writing agents — the same transform, every run.

The problem it owns

An agent's tools return big lists of records — search hits, file reads, API responses — far too verbose for a context window. Today the cutting-down is usually code the model improvises fresh on every run. That improvisation drifts: different field choices, different truncation, different survivors, run after run. The pipe that feeds the agent is itself a source of nondeterminism.

contexel replaces the improvisation with a small set of vetted, versioned stages applied at the tool → context boundary. Same input, byte-identical context, every run — a context contract instead of a nightly rewrite.

Deterministic

Byte-identical, proven

The benchmark hashes the same contract's output across fresh interpreters with different PYTHONHASHSEED values — same SHA-256 every time, even when raw tool output carries fresh request ids.

Token-aware

Budgets that hold

Stages reason in tokens, not rows. trim_to_budget and truncate_field enforce the limits you set; counting is pluggable (heuristic, tiktoken, your own) and encoding-aware.

Evidence-ranked

rescore, not trust

A BM25-style lexical scorer computed inside the batch derives relevance from the records' own text, so ranking no longer inherits the search tool's score quality.

Zero dependencies

Plain Python, 13 ms import

No embeddings, no model calls, no packages. Six pipeline stages plus a merge combiner, function composition, an opt-in trace — auditable in an afternoon.

The record, at a glance

Measured on the full CodeSearchNet python/test split — 28,047 real search-tool records. The outcome columns (Recall / Compliance / Useful) come from 100 ground-truth episodes at a 1,500-token budget where each library acts only through its own operations; ms @28k is the separate shared-glue canonical task. Details and method on the benchmarks page.

Implementation	Native ops	ms @28k	Recall (strong/weak signal)	Budget compliance	Useful tokens	Deps
contexel	6/6	59	100% / 100%	100%	100%	0
hand-written glue	0	27	93% / 32%	100%	100%	—
toolz	2	42	100% / 100%	0% (92× budget)	100%	0
langchain-core	1	139	30% / 30%	100%	89.8%	25
llama-index-core	0	516	100% / 100%	0% (129× budget)	74.2%	60

Read it honestly: contexel is the only row green across every column — and it is the slowest budget-compliant row because it does the most work: deriving relevance from ~1 KB of evidence per record on top of projecting, deduping, clipping, ranking, and budgeting. Every faster row is faster by doing less. All rows are negligible next to a single model call.

Sixty seconds of code

from contexel import select, dedupe, rescore, truncate_field, rank, trim_to_budget, trace

with trace() as t:
    r = select(raw, ["title", "url", "snippet", "published"])
    r = dedupe(r, key="url")
    r = rescore(r, query=user_query, fields=("title", "snippet"))
    r = truncate_field(r, "snippet", max_tokens=120)
    r = rank(r, by="score", desc=True)
    r = trim_to_budget(r, max_tokens=1500)

print(t.report())   # what every stage removed — inspectable, not silent

Continue with getting started, or see how the pieces fit.

contexel — the context element