Getting started

Install

pip install contexel              # pure Python, zero dependencies
pip install "contexel[accurate]"  # + tiktoken, for exact token counts

From source (also how you get the benchmark suites):

git clone https://github.com/maheshvaikri-code/contexel && cd contexel
pip install -e .                  # the library
pip install -e ".[accurate]"      # + tiktoken
pip install -e ".[benchmarks]"    # + everything the benchmark suites use

Shape your first tool output

Everything operates on list[dict] — the shape tools already return. Compose stages inline, and wrap them in trace() to see exactly what each one removed:

from contexel import select, dedupe, rescore, truncate_field, rank, trim_to_budget, trace

raw = search_tool(query=user_query)          # a batch of verbose results

with trace() as t:
    r = select(raw, ["title", "url", "snippet", "published"])
    r = dedupe(r, key="url")
    r = rescore(r, query=user_query, fields=("title", "snippet"))
    r = truncate_field(r, "snippet", max_tokens=120)
    r = rank(r, by="score", desc=True)
    r = trim_to_budget(r, max_tokens=1500)

print(t.report())    # per-stage: records in → out, tokens before → after
context = r          # only this distilled result enters the model's context

Audit the shaping

For a governance record — which policy version shaped this context, and exactly which records each stage dropped — open the trace with an id field:

with trace(id_field="url") as t:
    context = shape(raw)          # any pipeline([...]) call

record = t.audit()   # policy fingerprints, per-stage dropped ids, token movement

Records lacking the id field are untracked, so keep it in every select when full attribution matters. Gate untrusted sources before relevance logic with allowlist (and optionally quarantine) — see the stage reference.

Pin the policy at the tool boundary

For a policy the model can never vary, attach it to the tool itself with @shaped — the context contract pattern:

from contexel import shaped, stage, select, dedupe, trim_to_budget

@shaped([
    stage(select, fields=["path", "line", "snippet"]),
    stage(dedupe, key=["path", "line"]),
    stage(trim_to_budget, max_tokens=2000),
])
def search_code(query: str) -> list[dict]:
    return repo.search(query)     # raw and heavy — shaped identically, every call

Where the contract lives

Pattern 1

Agent sandbox

Programmatic tool calling: the sandbox imports the stages, the model composes them. examples/programmatic_tool_calling.py

Pattern 2

Tool wrapper

@shaped on the tool; the model never sees raw output. examples/tool_boundary_wrapper.py

Pattern 3

MCP server

Shape responses before they leave the server. examples/mcp_server.py

Pattern 4

Under a framework

A records ⇄ Document adapter for LangChain / LlamaIndex. examples/framework_adapter.py

Pattern 5

Shape, then encode

Price budgets in the encoding that actually enters context — e.g. an ISON table. examples/ison_boundary.py

Full example

Reference agent

A complete code-execution agent, no API key needed: python -m reference_agent. Its tests prove byte-identical context across runs.

Token accounting

Two pluggable axes — process-wide setters, plus a context-local overlay for concurrent tenants:

from contexel import tokens

tokens.use_tiktoken()          # exact where tiktoken covers the model
tokens.set_tokenizer(fn)       # or your provider's own counter
tokens.set_serializer(fn)      # price records in your boundary's encoding

with tokens.scoped(tokenizer=tokens.tiktoken_tokenizer()):
    shaped = contract(records)  # this task/tenant only; others untouched

Budgets are encoding-relative: a record's cost is the token count of its serialized text. If your boundary emits something other than JSON, plug that in so trim_to_budget prices what actually ships.

Next: the stage reference, or the evidence.