Put a deterministic Gate between model output and registered effects

01 — The problem

A hijack hidden in data

An LLM agent that reads untrusted data — an email, a web page, a tool result — can be hijacked by instructions hidden in that data and then act on them: leak your contacts, send mail, call tools on your behalf. This reference implementation places a deterministic authorization boundary on registered effects; it does not prove injection is unable to cause an unauthorized effect.

The mental model. Treat the model as a brilliant but untrusted contractor. The root planner receives no raw tool output. Quarantined reasoners may process untrusted data, but only in host-assembled contexts, without direct effect authority, and their outputs retain provenance.

The pattern fixes a topology, not a safety property: it places mediation and verification at two boundaries. The root planner is isolated from tool results. Delegated sub-planners see untrusted content under reduced grants; every registered tool invocation still passes its Gate. Whether that Gate's policy is correct is on the host.

02 — Why this works

Construction, not detection

Detection-based defenses try to recognize malicious prompts with filters, classifiers or guard models, and may miss novel phrasing. This kernel addresses a different question: regardless of phrasing, can registered model-driven effects reach a callable without deterministic authorization? That is a wiring question, not an impossibility proof.

03 — The two invariants

No trusted reasoner

Everything reduces to two rules the architecture never breaks. There is no trusted model in the system: two reasoners at differentiated privilege are both untrusted, and the trusted core is a deterministic interpreter plus a capability/provenance gate.

04 — The components

Each part, and what it does

The system is small and every seam is explicit. The reasoners are untrusted; the interpreter and Gate form the deterministic core. Host policies, schemas and adapters remain trusted configuration and code. Module paths point into src/reasoning_kernel/.

05 — The flows

How data and effects move

The trusted query becomes a plan; untrusted data is quarantined into typed values; every registered effect funnels through the single Gate. Each reasoner sees only its host-assembled context — Invariant A; the Gate mediates the effect path — Invariant B.

1. Flow A · plan & execute Trusted query → P-LLM → Plan → Interpreter The trusted path. The root planner sees only the controlled query and emits a typed plan; the interpreter runs it in a fixed order. Its plan does not branch on tool output. A delegated sub-planner may choose a child plan from untrusted content, but only under a reduced grant.
2. Flow B · quarantine Untrusted data → Q-LLM → tainted value How untrusted content enters. A q_parse step routes one piece of untrusted content through the Q-LLM into a typed value against a fixed schema. The result carries a taint label and cannot directly invoke a tool.
3. Flow C · effect & gate Tool effect → EffectDispatcher → Gate → commit The single choke point. A tool step is the only way to reach a side effect. The dispatcher checks the Gate (capability + schema + provenance) before the callable runs. Tainted data reaches a WRITE only if your declassifier deliberately allows it — and that decision is traced.
4. Flow D · delegate & combine subkernel (reduced grant) · merge (join of provenances) Composition stays clamped. A subkernel step runs untrusted content in an inner kernel at a clamped, smaller grant — an injection there is confined to what that grant permits. A merge step folds several reads into one value labelled with the join of their provenances. Authority only shrinks; taint only grows.

06 — What a run looks like

Two fixture attacks, blocked in the demo

“Summarize my latest email and send it to me” becomes a typed, four-step plan under RecipientIsUserPolicy and a scripted planner. Each step has one job, and only the last one can cause an effect:

read_inbox → q_parse (summarize the body) → const (my own address) → send_email

1. read_inboxReads the inbox through a tool. The returned email is untrusted — it carries a third-party taint label from here on.
2. q_parseSummarizes the body via the Q-LLM into a typed value. The body's hidden text is parsed as data; it never reaches the planner.
3. constYour own address, derived from the trusted query — not from anything the email said.
4. send_emailThe only effect. The Gate checks it: a summary of your own mail, sent to you, passes.

Run it yourself with just demo — the trace prints each gate decision and why.

07 — No effect bypasses the Verifier

Wiring, not an impossibility proof

Invariant B is not a convention to remember — it is forced by how registered tools are wired. Three facts, together, leave no other registered path to an effect. Side channels, host-owned callables outside the registry, and tools that exceed their spec are out of scope:

1. Tool callables live only in ToolRegistry, handed only to EffectDispatcher; the interpreter never holds one.
2. EffectDispatcher cannot be constructed without a Gate, and dispatch checks it unconditionally before the callable runs.
3. ToolCallStep is the only step kind that invokes a tool callable, through the dispatcher. A sub-kernel may execute its own tool steps through a reduced Gate and the same dispatch path.

08 — The plan IR

Five step kinds

A root Plan is a forward-only DAG of five step kinds and cannot branch or loop on tool output. A delegated sub-planner may choose its child plan from untrusted content under a clamped grant. Only one step kind can touch the outside world, always through the Gate.

09 — Embedding the kernel

Start with an operational session

RunSession is the bounded, single-use entry point. It wires a persistent audit sink, an isolated store, operational limits and the same explicit policy and adapter seams. Low-level Interpreter wiring remains available for specialized integrations.

from reasoning_kernel import (
    FakeProvider, PLLM, QLLM, RunContext, RunId, RunSession,
    SQLiteTraceSink, TrustedQuery,
)
from reasoning_kernel.demo.email_exfil import CLEAN_BODY, benign_plan, make_world
from reasoning_kernel.tools.demo_mail import (
    DEMO_GRANT, Q_SCHEMAS, EmailSummary, RecipientIsUserPolicy, build_registry,
)

ctx = RunContext(run_id=RunId("unique-request-id"), user="user@example.com",
                 query=TrustedQuery(text="Summarize my latest email and send it to me"))
provider = FakeProvider({"Plan": benign_plan(ctx.run_id),
                         "EmailSummary": EmailSummary(text="OK")})
with SQLiteTraceSink("audit.sqlite") as sink:
    session = RunSession(
        ctx=ctx, registry=build_registry(make_world(CLEAN_BODY)), grant=DEMO_GRANT,
        declass=RecipientIsUserPolicy(), planner=PLLM(provider, grant=DEMO_GRANT),
        quarantine=QLLM(provider), q_schemas=Q_SCHEMAS, sink=sink,
    )
    result = session.run()

Produce repeatable host evidence

Pin the package version in CI, then run the fixed profile that matches your integration. gate-v1 covers a verifier used before another pipeline; operational-v1 covers a complete RunSession host. Reports are sanitized observations from trusted host code, not signed safety attestations.

pip install capability-reasoning-kernel==0.6.4
reasoning-kernel-conformance my_agent.conformance:build_suite --output conformance.json

10 — Glossary

The vocabulary, in one place

The terms used above, defined plainly. Section references (e.g. §5.4) point to the CaMeL paper.

P-LLM / Q-LLM

The two untrusted reasoners: the privileged planner (emits a typed Plan) and the quarantined parser (turns untrusted content into typed data, with no tool access).

Taint / provenance

Every value carries a ProvenanceLabel recording where it came from (sources), where it may flow (readers), and whose data it is (subjects).

Join

Combining values combines their labels conservatively (union of sources, intersection of readers, union of subjects), so taint only ever increases.

Quarantine

Routing untrusted content through the Q-LLM, which turns it into typed data and cannot launder its taint.

Capability / grant

A host-issued permission a tool requires; a run holds a fixed CapabilitySet (its grant), and a sub-kernel's grant can only ever shrink.

Declassifier

The single deterministic seam (DeclassPolicy) that may let tainted data into a WRITE — the one place trust is deliberately relaxed.

Gate

The deterministic verifier every effect passes through: capability + schema + provenance.

11 — Threat model & limits

Reduction under assumptions, not an absolute claim

The pattern localizes the trust problem; it does not dissolve it. CaMeL-style separation reduces the injection-to-effect risk when planner input is trusted, the capability policy is correct, and tools enforce their spec. The full discussion, including which tests exercise the policy, is in the README.

• Assumptions. Trusted TrustedQuery, a correct grant and DeclassPolicy, tools that do only what their ToolSpec declares, no side channels, intact host TCB.
• Conformance ≠ safety. A pass-through declassifier conforms yet protects nothing. The pattern fixes a topology; the policy carries correctness.
• Non-goals. Side channels; policy bugs; a compromised planner relative to what the policy already allows; data exfiltration via allowed tools; formal impossibility proofs; provider confidentiality; rollback; Python sandboxing.
• Verification determinism is a discipline, not a typed invariant. Determinism is required of the declassifier, not enforced on it.
• Root control flow is static. The root DAG cannot branch on tool output; delegated child planning may depend on untrusted content only under a reduced grant.
• No atomicity / rollback, object-level taint. A committed effect is real even if a later step fails; labels cover whole values (the join over-approximates — strictly safer).

Get started

Read it, fork it, conform to it

It is a tested reference implementation and spec, not a turn-key security product or an independent audit. The working paper (technical note) states the topology, threat model, and limits — no peer-review or safety-certificate claim, and no Zenodo DOI until one is assigned. Install it, run the worked demo, then execute the versioned conformance profile against your own host.