# onetrace

> onetrace records one hash-chained receipt per stage of a Python pipeline (an AI/RAG/agent pipeline, or any other), so that anyone can verify the record without a network call and compare two runs stage by stage. `diff` and `localize` name the first stage that changed and the setting or tool version behind it.

Key facts for a coding agent:

- Python 3.10+. The console (`onetrace-console`) needs Python 3.12+, so start with 3.12 to use both. Install: `pip install onetrace` (SDK and verifier). This needs onetrace 0.2.0 or later: check with `python -c "import onetrace, onetrace_verify; print(onetrace.__version__, onetrace.__file__)"`, and if it prints an older version or a path outside the project's environment, use a fresh virtual environment. Signature and anchor checks: `pip install "onetrace-verify[crypto]"`.
- Instrument existing code with two decorators: `@ot.run` on the function that is one run, and `@ot.stage(name, ...)` on each step. Nothing else in the code changes. Called outside a run, a decorated stage is a plain function, so tests keep working.
- What a run folder holds: each stage's outputs (which can include documents, prompts and answers), and fingerprints (SHA-256 digests) of its inputs. Treat run folders like logs that may contain sensitive data. onetrace sends no telemetry and makes no network call during a run.
- See what changed: `onetrace diff runs/A runs/B --text` shows the first stage that differs and the text that changed in its outputs, checked against the receipts first.
- Verify a run: `onetrace-verify --require-artifacts runs/<run_id>` (exit 0 = record intact). Compare two runs: `onetrace diff runs/A runs/B`.
- Meaning is the human's: the trust class of what a stage reads and of what it returns, and whether the stage can be re-run, are never guessed. If omitted, the most cautious value is recorded (`externally-sourced`; not re-derivable) and the field is listed as `undeclared` on that stage. An agent must leave them omitted and ask the human, not fill them in. The same holds for `keep_inputs=`, which stores an input's text in plain form in the run folder.
- Settings that explain a change (chunk size, top-k, the model and the settings the code sends with it) go in `ot.pkg(..., config={...})` or `ot.constant(key, value)`. Without them, `localize` can say a stage's output changed but not why. `config` is recorded as one digest, so a comparison says the config changed, not which key: put a literal from the code in `config=`, and a value from a flag, an argument or a variable in `ot.constant`, so a comparison names it.
- `ot.pkg(id, package, ...)` reads `package`'s version from its installed metadata. For your own code, that is your project's distribution, installed (`pip install -e .`; a project of scripts needs a minimal `pyproject.toml` first).
- A run declares the stages its own module defines when it is imported. List every stage in `@ot.run(stages=[...])` when one is defined in another module or decorated inside a function while the run is running.
- A step that is a method on an object (a retriever index), or that receives an SDK client holding a key, gets a thin wrapper: a small inner function that takes only plain values, uses the object or client from the enclosing scope, and is the decorated stage. The key never passes through a decorated function.
- A folder of documents: `files=` refuses two files with the same name in different subfolders. Record one SHA-256 over the folder's sorted relative paths and bytes with `ot.constant` instead.
- RAG pipelines: an ingest run and a query run each get `@ot.run`. Link the query run to what it reads (`ot.corpus_from(...)`, or the external index's name and version recorded with `ot.constant`), or a comparison can't say that the index changed. With the link, `onetrace diff A B --follow-corpus` follows a changed link one hop into the two ingest runs and names where they first differ. A stage that groups steps (embed + search) records every hidden step's settings.
- DAGs are supported: pass one stage's return value to another, the same object and whole, and the edge is recorded; a part or a copy of it is an ordinary input with no edge. Fan-out and fan-in work. A stage runs once per run unless declared `repeats=True` (a loop or a retry; calls numbered #1, #2, …); overlapping calls of the same stage are each named, `stage_fn.instance("en")(...)`.
- Switch off with `ONETRACE_DISABLE=1`; removing the decorators restores the original code exactly.
- What a verified record does not show: that the output was correct; why a model wrote what it wrote; anything about code that was not decorated.

## Instrumenting a codebase

- [Agent recipe: instrument an existing pipeline](https://oneproof.dev/onetrace/agents/instrument.md): step-by-step instructions for a coding agent, with the rules it must not break (fetch the file itself, not a summary)
- [AGENTS.md section to paste into your repo](https://oneproof.dev/onetrace/agents/AGENTS-section.md): keeps future agent edits consistent with the instrumentation
- [Quickstart](https://oneproof.dev/onetrace/quickstart.md): install, two decorators, run, verify
- [Tool-using agent](https://oneproof.dev/onetrace/tool-using-agent.md): an agent loop recorded step by step, with a stand-in model: how the steps link, the trust class of each value, and comparing runs with a hosted model
- [Decorators](https://oneproof.dev/onetrace/manual.md#16-decorators): what a decorated stage stores, the fields you didn't state, repeated and concurrent calls, `files=`, switching off, encoders, the corpus link
- [Numbers and booleans](https://oneproof.dev/onetrace/manual.md#1616-numbers-and-booleans-rule-m-for-settings-rule-v1-for-values): how settings (rule M) and decorated values (rule V1) are encoded
- [`@ot.run`'s defaults](https://oneproof.dev/onetrace/manual.md#1617-otruns-defaults): the policy, manifest, declared stages, intake stage and run folder a decorated run uses when you don't say
- [Error messages and their fixes](https://oneproof.dev/onetrace/errors.md): every refusal, why it happens, the one-line fix

## Checking and comparing runs

- [Verifying a run](https://oneproof.dev/onetrace/manual.md#8-verifying-a-run): `onetrace-verify`, exit codes, `--require-artifacts`, `--trust`
- [diff and localize](https://oneproof.dev/onetrace/verdicts.md): the five verdicts (`same`, `FIRST DIFFERENCE`, `downstream`, `reconverged`, `COULD NOT CHECK`) and what each means; and a run's verdict against a reference — `same output`, `output differs`, `setting differs`, `input differs`, `slower than its range`, `could not check` — from onetrace verdict and onetrace-verify --reference
- [References](https://oneproof.dev/onetrace/references.md): a sealed record of a known-good run a later run is compared against — per step the output digests, settings, data inputs and a timing range, with the rule; made with onetrace reference make, shown with onetrace reference show
- [Variation across repeat runs](https://oneproof.dev/onetrace/variation.md): `repeat_of`, `may_vary`, onetrace repeat, and onetrace profile — how often each step's output differs across a set, and whether a fix holds
- [Streaming the record](https://oneproof.dev/onetrace/streaming.md): `@ot.run(stream_to=URL)` / `ONETRACE_STREAM_TO` sends each receipt as it is written; it changes no receipt and adds nothing to the chain
- [Storage: a run in a bucket](https://oneproof.dev/onetrace/storage.md): ONETRACE_RUNS=s3://bucket/prefix uploads a sealed run to an S3-compatible bucket behind the optional onetrace[s3] extra; the index object, onetrace sink retry and onetrace sink verify; a bucket verifies nothing, the reader verifies on every read
- [onedoor decisions](https://oneproof.dev/onetrace/decisions.md): `gated_stages`, `ot.decision`, the receipt's `authorization` block, and the five verifier rows (`--decisions`, `--permit-issuers`). onetrace never imports onedoor
- [The span context](https://oneproof.dev/onetrace/span-context.md): a step's W3C trace id and span id, and the run's `traceparent`, read from an active OpenTelemetry span or `TRACEPARENT`; no OpenTelemetry package required
- [Nesting](https://oneproof.dev/onetrace/nesting.md): `parent`/`children` on a stage, `parent_run`/`parent_stage` on a nested run, the depth limit, the comparison's descent, and the `--nested-run` verifier row
- [The model-call record](https://oneproof.dev/onetrace/model-call-record.md): one canonical record per model call, beside the stage receipt, behind the optional `onetrace[model-call]` extra — the five states and what each field observed or declared, the per-step capture as a separate model-generated output the record names by digest, the comparator's three results, the verifier's rows, `ot.model_from`, and what it does not claim
- [What onetrace does not claim](https://oneproof.dev/onetrace/non-claims.md)
- [Suggested explanations](https://oneproof.dev/onetrace/explain-format.md): `onetrace-explain`, a separate package: what it sends to the model you configure, how each claim is checked against the record before it is shown, and what it writes. A suggestion about a record, never a reading of it

## Optional

- [Recorder API](https://oneproof.dev/onetrace/manual.md#5-integrating-onetrace-into-your-code-step-by-step): full control, without decorators
- [Anchoring a run](https://oneproof.dev/onetrace/manual.md#15-anchoring-a-run)
- [Signing a run](https://oneproof.dev/onetrace/manual.md#17-signing-a-run): a key the human holds, `sign_with=` at close or `onetrace sign` afterwards, and `onetrace-verify --trust`
- [Receipt format](https://oneproof.dev/onetrace/format.md): the stage-receipt format and its Internet-Draft
- [Worked examples with frameworks](https://oneproof.dev/onetrace/README.md#worked-examples-with-frameworks): plain Python, LangChain, LlamaIndex, Langflow

<!-- https://oneproof.dev/onetrace/agents/instrument.md -->

> *For people: this page is written for your coding agent, not for you to follow. Hand it over with the prompt or the agent kit at <https://oneproof.dev/developers.html#agent>, or start at <https://oneproof.dev/get-started.html>.*

# Recipe for coding agents: instrument an existing Python pipeline with onetrace

You are adding onetrace to a codebase you did not write. Your job is to add **recording**, not to change **behaviour**. When you finish, the pipeline must produce exactly the same results as before, and every run must leave a record that `onetrace-verify` accepts.

Read all of this before editing. The rules in "Never" are hard rules. Fetch this file itself, as raw text, from https://oneproof.dev/onetrace/agents/instrument.md, and work from it, not from a summary of it.

**The rules, in short** (each is explained below):

1. Add recording. Change nothing the pipeline computes: no signature, output or logic changes.
2. Leave out `trust=`, `output_trust=`, `rederivable=` and `note=`, and ask the human about each.
3. No secret, token or key in a decorator, `config=`, `ot.constant` or a decorated function's arguments: a step that takes a client holding a key gets a thin wrapper (§3).
4. Never turn on signing, decorate library code, add network calls, anchoring or telemetry, or edit CI files unasked.
5. List in `@ot.run(stages=[...])` every stage the run's own module doesn't define when it is imported.
6. A literal setting goes in `config=`; a setting from a flag, an argument or a variable goes in `ot.constant`.
7. Use the real services. A stubbed run says so in its run id (`ONETRACE_RUN_ID=stub-…`), and isn't proof.
8. Run every run in full, verify it with `onetrace-verify --require-artifacts`, and compare two runs.
9. End with the report in §7, including your questions for the human.
10. Decorate the function production calls. Never write a separate runner or harness that calls the stages its own way: the record must be of the path production takes (§1).

## 0. Before you start

- Confirm Python ≥ 3.10 and that the project's tests pass **before** any change. Record the test command and its result. If the tests don't pass, stop and tell the human.
- Install: `pip install onetrace` (add it to the project's dependency file the way the project already does it: `pyproject.toml`, `requirements.txt`, lock file).
- **This needs onetrace 0.2.0 or later.** Check which onetrace Python actually imports:
  ```
  python -c "import onetrace, onetrace_verify; print(onetrace.__version__, onetrace.__file__)"
  ```
  If the version is below 0.2.0, or the path isn't in the environment the project runs in, make a fresh virtual environment and install onetrace there.
- Run `onetrace doctor`. Fix what it reports before going on.
- **Your own code needs an installed distribution.** `ot.pkg` reads the version of the package you name from its installed metadata. For a project that isn't installed, `ot.pkg` is refused when the stage is decorated, which for a stage at module level is when its module is imported (`package 'my-pipeline' is not installed, so its version can't be read`). For a project of scripts with no `pyproject.toml`, add a minimal one, then run `pip install -e .`, and tell the human you added it:
  <!-- not executed: a file to save, not a command; tests/test_agent_recipe_examples.py builds it -->
  ```toml
  [build-system]
  requires = ["setuptools>=77"]
  build-backend = "setuptools.build_meta"

  [project]
  name = "my-pipeline"
  version = "0.1.0"

  [tool.setuptools]
  py-modules = []
  ```

## 1. Find the run

A **run** is one call of one function that does the whole job once: answers one question, processes one document batch, handles one request. Find that function (often `main`, `run`, `handle`, `answer`, or the function the CLI or API route calls).

- If there are two kinds of run (for example an **ingest** job that builds an index, and a **query** that uses it), each gets its own `@ot.run`.
- **Link the query run to what it reads.** If the query reads an index that an ingest run built:
  - when the index is built before the run, by the project's own code, record that build as an **ingest run** (its own `@ot.run`, with its own stages), and link each query run to it with `@ot.run(corpus=ot.corpus_from(<ingest run folder>, stages=[<the stages that read it>]))`. The ingest run is verified when the query run starts. Keep the ingest runs in the same folder as the query runs: `onetrace diff A B --follow-corpus` then follows a changed link one hop into the two ingest runs and names the stage where they first differ, and why (manual, section 16.13);
  - when the index lives in an **external service** (Qdrant, Pinecone, Weaviate, a database), record its identity on the stage that reads it with `ot.constant(...)`: the collection or index name, and the ingest run id or version that built it, if the code knows it. Then list the service as a question for the human ("retrieve reads collection X in Qdrant, outside the run: how should it be treated?").

  Without a link, a later comparison can say *that* the retrieved chunks changed, but not that the index behind them changed.
- **Decorate the function production calls**, where it is: the one the CLI, the API route or the job runner already calls. Never write a new runner, script or test harness that calls the stages in its own way and decorate that instead: its runs would record a path production never takes. `onetrace doctor --entrypoint <file>` (step 4 in §6) checks the most recent run was recorded through the file that holds that function.
- If you can't tell which function is one run, stop and ask the human. Do not guess.

## 2. Find the stages

A **stage** is a function the run calls that does one distinct step whose result matters. Typical stages in AI pipelines:

| Step | Typical `kind` |
|---|---|
| load / read documents | `reader` |
| clean, convert, normalise | `transformer` |
| split into chunks | `transformer` |
| embed | `embedder` |
| build or update an index | `indexer` |
| retrieve | `retriever` |
| build the prompt | `formatter` |
| call a model | `model-call` |
| parse the model's output | `parser` |
| act on it (write a file, call an API) | `actuator` |

Rules for choosing:

- Only functions **defined in this repository**. Never decorate a library function. If a single library call does several steps (for example `VectorStoreIndex.from_documents(docs)`), it is **one** stage, unless the human asks you to split it into small functions of the project's own.
- Prefer **fewer, meaningful stages** over many tiny ones. Five to ten is typical. Starting with just one stage is fine.
- A decorated stage must not call another decorated stage. Decorate the outer one or the inner ones, not both.

## 3. Add the decorators

```python
import onetrace as ot

@ot.run(run_dir="runs/{run_id}")
def answer_question(question: str) -> str:
    docs = retrieve(question)
    return generate(question, docs)

@ot.stage("retrieve",
          instrument=ot.pkg("bm25-retriever", "rank_bm25", kind="retriever",
                            config={"top_k": 4}),
          files=["data/corpus.json"])
def retrieve(question: str) -> list[str]:
    ...

@ot.stage("generate",
          instrument=ot.pkg("model-call", "openai", kind="model-call",
                            config={"model": "gpt-4.1-mini"}))
def generate(question: str, docs: list[str]) -> str:
    ...
```

- **`stages=[...]`:** by default a run records the stages its own module defines when it is imported. List every stage in the run's order, `@ot.run(run_dir="runs/{run_id}", stages=["retrieve", "generate"])`, when any stage is not one of those: a stage defined in another module, or a stage decorated inside a function while the run is running (a thin wrapper, below). A stage the run calls without that is refused, and the message names both fixes.
- **One import** per file you touch: `import onetrace as ot`. Change nothing else in the file: no reformatting, no renames, no signature changes, no logic changes. The thin wrappers and the folder digest below add code without changing what the pipeline computes.
- **`ot.pkg(id, package, kind=..., config=...)`**: `package` is the installed distribution that does the work (its version is read from its installed metadata; never type a version). `id` is a short stable name for the tool. For a stage that is your own code, `package` is your project's own distribution, as its `pyproject.toml` names it, installed (§0).
- **`config=` or `ot.constant`:** `config` is recorded as one digest, so when it changes a comparison says the stage's config changed, not which key. Put a setting in `config=` when it is a **literal value in the code**, copied exactly. A setting whose value comes from a flag, an argument, a settings file or a variable goes in `ot.constant("chunk_size", chunk_size)` inside the function, and a comparison then names it by its key (`constant chunk_size differs`). **Never record the value of an environment variable that holds a key or token.**
- **Model settings:** record the settings the code actually sends with the call: the model name, and whatever else it passes, each from where the code takes it. Don't add a setting the code doesn't send, to the call or to the record: a model can reject a parameter it doesn't support.
- **`files=[...]`**: files the stage reads from disk (corpus, index, prompt templates). Write each path as the code opens it: a relative path is read from the directory the pipeline runs in, not from the repository root.
- **A folder of documents:** `files=` records each file by its name, so two files with the same name in different subfolders are refused when the stage is decorated. For a folder, record one digest over its files' sorted relative paths and bytes, with `ot.constant`:
  ```python
  import hashlib
  from pathlib import Path

  import onetrace as ot

  def folder_sha256(root) -> str:
      root = Path(root)
      h = hashlib.sha256()
      for p in sorted((q for q in root.rglob("*") if q.is_file()),
                      key=lambda q: q.relative_to(root).as_posix()):
          h.update(p.relative_to(root).as_posix().encode("utf-8") + b"\0")
          h.update(hashlib.sha256(p.read_bytes()).digest())
      return h.hexdigest()

  @ot.stage("load", instrument=ot.pkg("loader", "my-pipeline", kind="reader"))
  def load(docs_dir: str) -> list[str]:
      ot.constant("docs_sha256", folder_sha256(docs_dir))
      ...
  ```
  A comparison then names the change as `constant docs_sha256 differs`.
- **A step that is a method on an object, or that receives an SDK client holding a key** (a retriever index's `search`, a function given `client`): don't decorate it. Add a **thin wrapper**: a small inner function that takes only plain values, uses the object or client from the enclosing scope, and is the decorated stage. The key never passes through a decorated function, and what the pipeline computes doesn't change. List the stages in `stages=[...]`, since they're decorated while the run is running:
  ```python
  import onetrace as ot

  MODEL = "your-model-name"

  def retrieve(index, question, k=2):
      @ot.stage("retrieve", instrument=ot.pkg("bm25", "my-pipeline", kind="retriever"))
      def search(question: str, k: int) -> list[str]:
          return index.search(question, k)
      return search(question, k)

  def generate(client, question, passages):
      @ot.stage("generate", instrument=ot.pkg("model-call", "my-pipeline", kind="model-call",
                                              config={"model": MODEL}))
      def call(question: str, passages: list[str]) -> str:
          return client.complete(MODEL, question + "\n\n" + "\n".join(passages))
      return call(question, passages)

  @ot.run(run_dir="runs/{run_id}", stages=["retrieve", "generate"])
  def answer(question: str) -> str:
      client = Client()        # the project's own client, holding its key
      return generate(client, question, retrieve(Index(), question))
  ```
  The object itself isn't recorded. Record what identifies it: the ingest run that built the index (`ot.corpus_from`, §1), or its name and version with `ot.constant`.
- **Values between stages** must be plain data: strings, bytes, ints, lists and dicts of these, pydantic models, dataclasses, numpy arrays. If a stage returns an object onetrace can't encode (an index object, a database client), don't change the function. Tell the human, and suggest either returning the saved index path or registering an encoder with `ot.encoder`.
- **Pass a stage's output on whole.** Two stages link only when the second is given the very object the first returned. Given a part of it (`reply["id"]`, a slice, a copy), the second stage records an ordinary input and no link. Don't restructure the code to change this; tell the human which stages aren't linked.
- **`ot.constant` after the model call is fine.** A setting known only from the reply (the model version it answered with) is recorded like any other: a stage's constants are written when it returns.
- **An agent loop** (the model asks for a tool, the tool runs, the model is called again): the model call and each tool are stages, each with `repeats=True`. The guide at https://oneproof.dev/onetrace/tool-using-agent.md shows the shape.
- **A stage called more than once in a run:** by default a stage runs once per run, and a second call is refused. For sequential repeats (a loop, a retry), declare the stage `repeats=True`: each call is numbered #1, #2, …. For calls that overlap (`asyncio.gather`, a thread pool), name each call with a stable name per branch instead, for example `retrieve.instance("en")(question)` and `retrieve.instance("fr")(question)`.

- **What a stage depends on, but its code doesn't show:** for example, a prompt template in another module, or a settings file. Record its version or content hash with `ot.constant`, so a change to it is named as the reason.
- **Everything a stage uses must reach it in a way the record sees:** as an argument, a `files=` entry, or `ot.constant`. A value the stage takes from a global, an enclosing function or `self` (the user's question, a settings object) is not recorded, and a comparison will then show the stage's output changing while none of its inputs changed. Don't restructure the code to fix this. Record the value with `ot.constant` (for text, its SHA-256, never the text itself), and tell the human.
- **A stage that groups several steps** (for example `retrieve` = embed the query + search the index) stays one stage. Don't refactor to split it. Instead, **record the settings of every hidden step on that stage**: the embedding model, top-k, the collection name, and any filters. That's what lets a comparison say *why* the stage's output changed.

## 4. Leave the meaning to the human

These fields say what a stage *means*, and only a person may set them:

- `trust=`: where the values a stage reads come from (`operator-authored`, `externally-sourced`, …);
- `output_trust=`: the same, for what the stage returns. The trust class covers outputs too: a decorated stage's output with no stated class is recorded with the most cautious one and listed as undeclared, like an input's;
- `rederivable=` and `note=`: whether running the stage again gives the same output (a hosted model usually doesn't);
- `keep_inputs=`: whether a stage keeps an input's text in plain form beside its digest, in the run folder (manual, section 16.19).

In a tool-using agent, ask about each tool's output: a lookup in a system the human runs is usually `operator-authored`, a web page or another party's API `externally-sourced`, and what a model wrote `model-generated`. The human decides each one.

**Do not set these.** Leave them out. onetrace then records the most cautious value (`externally-sourced`; not re-derivable) and lists the field as `undeclared` on that stage, so everyone can see nobody stated it. At the end, list them as questions for the human (step 7), with what you observed, for example: *"`generate` calls a hosted model: is it re-derivable? (usually no)"*.

## 5. Check behaviour is unchanged

1. Run the project's test command. The same tests must pass as in step 0, with the same results. If anything differs, undo your change to that function and report it.
2. **Use the real services.** If a service the pipeline needs is down (a vector database, a model API), stop and tell the human. Never substitute stubs or mocks silently. A run made with stubs must say so in its run id, and it doesn't count as proof. If the human agrees to stub a model API, do it on the command line, with no mock code inside the pipeline: start a local fake endpoint, point the client library at it with its base-URL environment variable, and give the run a stub id, in one line:
   <!-- not executed: needs a fake endpoint listening on that port, and the project's own entry point -->
   ```
   ONETRACE_RUN_ID=stub-query-1 OPENAI_BASE_URL=http://127.0.0.1:8000/v1 python -m app "a question"
   ```
   `OPENAI_BASE_URL` is the `openai` package's variable; other client libraries name their own. Each run needs a new id: a run folder that already holds anything is refused, and `a:b` and `a-b` name the same folder.
3. Run **every run function in full** (for a RAG pipeline, the whole ingest run *and* the whole query run, all stages), the way the project normally runs it or through the test that exercises it. A folder appears under `runs/` for each, and when each run closes it prints that folder and the command that verifies it, on stderr. A shortened demo run that skips stages doesn't count.
4. Verify it:
   <!-- not executed: <run_id> stands for the folder your run wrote -->
   ```
   onetrace-verify --require-artifacts runs/<run_id>
   ```
   Exit code 0 means the record is intact. The verifier's first line, on stderr, says so (`record intact: …`), or names the first failing row (`record NOT intact: …`), and under it a `fix (…)` line for each failing row says what to change and which section of the errors page explains it. Don't set `ONETRACE_QUIET=1` while you check: it silences those lines. Anything else: read that row and its `fix (…)` line; the errors page explains each one. `onetrace doctor` checks the setup and says whether the most recent run verifies, not why a row failed. Then run `onetrace doctor --entrypoint <the file holding the decorated run function>`: it fails if the run was recorded through any other file. Fix the instrumentation, not the pipeline.
5. **Run it a second time with nothing changed, and compare:** `onetrace diff runs/<first> runs/<second>`. Every stage should read `same`, except stages that call a hosted model, or read a service that changes by itself.
   - If any other stage differs, **stop and find out why before anything else.** Look first for ids generated fresh each run (uuid), timestamps written into outputs, absolute paths, and unordered sets or dict iteration.
   - A stage that calls a hosted model reading differently is expected. Don't stub it to make the comparison read `same` (step 2). The console's Compare page then also names the first later stage with a recorded change.
   - Report it to the human with the stage and the field.
   - Until it's fixed, every comparison will blame that stage first and hide the real change.
6. Only then make one deliberate change (for example chunk size) and compare again. The first difference must be the stage you changed. `onetrace diff` and `onetrace localize` name a setting recorded with `ot.constant` by its key ("constant chunk_size differs"); for a setting in `config=` they say the stage's config changed, not which key. To have the key named, record the setting with `ot.constant`.
7. For a query pipeline, also compare two **different** questions. `retrieve` and the model's answer should normally differ. If they don't, report it: it usually means something was stubbed or cached. Also check that every stage that uses the question (retrieve, prompt building) lists it among its inputs.

## 6. Optional, only if the human asks

- A CI check: follow the manual's section 11, which verifies the run with `onetrace-verify` and compares it with a known-good run with `onetrace diff`.
- Signing: **not yours to turn on.** If the human wants runs signed, they make the key (`onetrace keygen`), keep it, and add `sign_with=` themselves; the manual's section 17 explains it. **Never create, print, copy or commit a private key yourself.**
- Add the onetrace section to the repository's `AGENTS.md` (see the AGENTS.md section page), so later edits keep the instrumentation consistent.

## 7. Report to the human

End with a short report:

- the run function(s) and the stages you decorated, file and line;
- the settings recorded per stage, and any you saw but could not record;
- **questions**: every `trust`, `output_trust` and `rederivable` you left undeclared, each with what you observed;
  - whether runs should be signed, and with whose key: signing is the human's choice, with a key the human holds (`sign_with=`, `onetrace keygen`, `onetrace sign`);
  - if a stage splits documents into chunks: onetrace can record a chunk index (`ot.chunks`) so a comparison can show which chunk changed, and with `keep_text=True` its text. That changes the function's return value and stores the corpus text in run folders, so it is the human's choice; ask;
- what is **not** covered: steps inside library calls, files read but not listed, code you did not decorate, external services the run reads or writes;
- **what changes the record can and can't explain.** Don't report coverage as "n of n stages": you chose the stages, so that number is always complete. Instead, list the likely changes and say, for each, whether a comparison of two runs would name the stage **and** the reason:

  | Likely change | Explained? | Why or why not |
  |---|---|---|
  | a source document edited | … | … |
  | chunk size or overlap | … | … |
  | the embedding model (at ingest, and at query time) | … | … |
  | top-k or retrieval filters | … | … |
  | the external index changed (re-indexed or edited by hand) | … | … |
  | the prompt template | … | … |
  | the model, or a setting the code sends with it | … | … |
  | the model simply answered differently | … | … |

  Add rows for anything specific to this pipeline. Where a row says "no", say what would fix it: a setting to record, or a link to add.
- the test result before and after, the `onetrace-verify` result, and the `onetrace diff` of two runs (§5).

## Never

- Never change what the pipeline computes, its function signatures, or its outputs.
- Never set `trust`, `output_trust`, `rederivable` or `note` yourself. That includes calls to hosted models and external services: describe what you saw, and ask.
  - If your onetrace version makes you pass a trust class, pick the most cautious one, and list every choice in your report as a question.
  - Never mark text a user typed as `operator-authored`.
- Never put a secret, token, key or password into a decorator argument, `config`, `ot.constant`, or a stage's arguments. If a function receives an API key, or a client holding one, as an argument, don't decorate it: decorate a thin wrapper inside it (§3), or tell the human.
- Never create a `Recorder` at module level, and never decorate library code.
- Never turn on signing: no `sign_with=` on `@ot.run` or `Recorder`, and no `onetrace keygen` or `onetrace sign`. Signing is the human's choice, with the human's key.
- Never add network calls, anchoring, or telemetry.
- Never edit CI workflow files unless the human asked.
- Never claim the record shows the output is correct. It shows what each decorated stage received and produced, and that the record hasn't been altered since.

<!-- https://oneproof.dev/onetrace/agents/AGENTS-section.md -->

<!-- Paste this section into your repository's AGENTS.md (or CLAUDE.md, .cursorrules, copilot-instructions.md). Edit the bracketed lines. -->

## onetrace (run records)

This repository records each pipeline run with onetrace: one verifiable receipt per stage, written under `runs/`.

- **Run functions:** [e.g. `app/pipeline.py:answer_question`] carries `@ot.run`. **Stages:** functions decorated with `@ot.stage`.
- **When you change a decorated stage:** keep the decorator. If you change a setting recorded in `config=` or `ot.constant(...)` (chunk size, top-k, the model and the settings the code sends with it), update the recorded value in the same edit. A literal in the code goes in `config=`; a value from a flag, an argument or a variable goes in `ot.constant`, so a comparison names it by its key. A setting that changes without being recorded makes a later comparison say *that* the stage changed, but not *why*.
- **When you change how an index is built, or which collection or index a query reads:** update the recorded link or constants (`ot.corpus_from(...)`, or the collection name and ingest version recorded with `ot.constant`) in the same edit.
- **When you add a step that matters to the result:** add `@ot.stage` following the recipe at https://oneproof.dev/onetrace/agents/instrument.md. Only functions defined in this repository; never library functions; a decorated stage must not call another decorated stage. A stage defined in another module, or decorated inside a function while the run is running, is listed in `@ot.run(stages=[...])`.
- **When you pass one stage's output to another:** pass the whole object it returned. A part or a copy of it is recorded as an ordinary input, with no link between the stages.
- **Do not set** `trust=`, `output_trust=`, `rederivable=`, `note=` or `keep_inputs=`. A person decides those. Leave them out and mention them in your summary as questions.
- **Never** put secrets, tokens or keys into decorator arguments, `config`, `ot.constant`, or a decorated function's arguments: a step that receives a client holding a key gets a thin wrapper, as the recipe shows. Don't add or change `sign_with=`, and never create or commit signing keys: signing is a person's choice, with their own key.
- **Do not edit or delete** anything under `runs/` or `baselines/`. A baseline changes only in a pull request a person approves.
- **After changing pipeline code, check:**
  <!-- not executed: the bracketed line is the project's own test command, filled in when pasted -->
  ```
  [project test command]
  onetrace-verify --require-artifacts runs/<latest run>
  ```
  If CI compares the run with a baseline (`onetrace diff baselines/golden runs/<run>`) and it reports a difference, the report names the first stage that changed; explain in the pull request whether that change was intended. Don't regenerate the baseline to make CI pass unless a person asked.
- **To switch recording off locally:** `ONETRACE_DISABLE=1`. Don't remove decorators to silence an error; fix the cause or ask.

<!-- https://oneproof.dev/onetrace/quickstart.md -->

# Quickstart

Add onetrace to a small pipeline in two steps: one stage first, then the rest. Then switch it off, and take it out. After the install, nothing here makes a network call; there's no model and no key, and it takes a few seconds.

Use a fresh virtual environment with Python 3.10 or later. The console (`onetrace-console`) needs Python 3.12 or later, so start with 3.12 to use both. Each command runs from the folder you save the files in.

<!-- quickstart: run -->
```
pip install onetrace
```

## Your pipeline

A pipeline in two modules: `steps.py` finds passages, `app.py` answers from them. Save both.

<!-- quickstart: file steps.py -->
```python
CORPUS = [
    "The warranty covers manufacturing defects for twelve months.",
    "Shipping delays are handled by the courier, not the warranty.",
    "Batteries have a separate six-month warranty.",
]


def retrieve(question: str) -> list[str]:
    words = set(question.lower().rstrip("?").split())
    return sorted(CORPUS, key=lambda p: -len(words & set(p.lower().rstrip(".").split())))[:2]
```

<!-- quickstart: file app.py -->
```python
from steps import retrieve


def answer(question: str, passages: list[str]) -> str:
    return passages[0]


def main(question: str) -> str:
    return answer(question, retrieve(question))


if __name__ == "__main__":
    print(main("How long does the warranty cover defects?"))
```

<!-- quickstart: run -->
```
python app.py
```

<!-- quickstart: output -->
```
The warranty covers manufacturing defects for twelve months.
```

## Step 1: one stage

Add one import and two decorators to `app.py`, and change nothing else: `@ot.run` on the function that does the whole job once, and `@ot.stage` on one step.

<!-- quickstart: file app.py -->
```python
import onetrace as ot
from steps import retrieve


@ot.stage("answer", instrument=ot.pkg("first-passage", "onetrace", kind="model-call"))
def answer(question: str, passages: list[str]) -> str:
    return passages[0]


@ot.run()
def main(question: str) -> str:
    return answer(question, retrieve(question))


if __name__ == "__main__":
    print(main("How long does the warranty cover defects?"))
```

`ot.pkg` names the tool a stage uses: a short id, the installed distribution whose version is recorded, and its `kind`. This demo isn't an installed project, so it names `onetrace`; in your own code, name the library that does the work, or your project's own distribution.

<!-- quickstart: run -->
```
python app.py
```

<!-- quickstart: output -->
```
The warranty covers manufacturing defects for twelve months.
```

The run wrote a folder under `runs/`, named by its run id. `onetrace doctor` checks the installation and verifies the most recent run:

<!-- quickstart: run -->
```
onetrace doctor
```

<!-- quickstart: output -->
```
...
6 checks; all passed
```

**What this one-stage run records.** Two receipts, not one:

- **`intake`**: `@ot.run` records the run function's arguments as a stage of their own. Here, the digest of `question`.
- **`answer`**: its arguments (`question` and `passages`), each as a SHA-256 digest of its encoding ([rule V1](manual.md#1616-numbers-and-booleans-rule-m-for-settings-rule-v1-for-values)), and its return value, stored as `return.json` in the run folder.

And what it doesn't:

- **`retrieve` is not recorded at all.** It isn't decorated, so the record has no step for it and no boundary in its place. Its result reaches `answer` only as an argument's digest. If the passages change, the record shows `answer`'s input changing, and nothing about why.
- **Nothing about meaning.** No stage says where its input came from (`trust`), where its output came from (`output_trust`), or whether it can be re-run to the same result (`rederivable`). onetrace records the most cautious value and lists each such field as `undeclared` on the stage. A person decides those; see [the manual's Decorators section](manual.md#16-decorators).

## Step 2: the other stage, in another module

Decorate `retrieve` in `steps.py`:

<!-- quickstart: file steps.py -->
```python
import onetrace as ot

CORPUS = [
    "The warranty covers manufacturing defects for twelve months.",
    "Shipping delays are handled by the courier, not the warranty.",
    "Batteries have a separate six-month warranty.",
]


@ot.stage("retrieve", instrument=ot.pkg("word-overlap", "onetrace", kind="retriever", config={"top_k": 2}))
def retrieve(question: str) -> list[str]:
    words = set(question.lower().rstrip("?").split())
    return sorted(CORPUS, key=lambda p: -len(words & set(p.lower().rstrip(".").split())))[:2]
```

A run records the stages defined in its own module ([`@ot.run`'s defaults](manual.md#1617-otruns-defaults) lists them all). `retrieve` is defined in another one, so list the run's stages, in order, on `@ot.run`. Without that list, the call to `retrieve` is refused, and the message names this fix. In `app.py`, change only the `@ot.run` line:

<!-- quickstart: file app.py -->
```python
import onetrace as ot
from steps import retrieve


@ot.stage("answer", instrument=ot.pkg("first-passage", "onetrace", kind="model-call"))
def answer(question: str, passages: list[str]) -> str:
    return passages[0]


@ot.run(stages=["retrieve", "answer"])
def main(question: str) -> str:
    return answer(question, retrieve(question))


if __name__ == "__main__":
    print(main("How long does the warranty cover defects?"))
```

<!-- quickstart: run -->
```
python app.py
```

<!-- quickstart: output -->
```
The warranty covers manufacturing defects for twelve months.
```

<!-- quickstart: run -->
```
onetrace doctor
```

<!-- quickstart: output -->
```
...
6 checks; all passed
```

Now `retrieve` has a receipt of its own. Its setting (`top_k`) is part of its instrument's recorded config, so a change to it shows as a changed config. `answer` records the passages as `retrieve`'s output: an edge from one stage to the next.

When a run closes, onetrace prints two lines to stderr: the folder it wrote, and the command that verifies it ([what a run says when it closes](manual.md#1618-what-a-run-says-when-it-closes-onetrace_quiet1)). `ONETRACE_QUIET=1` turns them off.

<!-- not executed: the run id differs on every run -->
```
onetrace: run written to runs/<run id>
next: onetrace-verify --require-artifacts runs/<run id>
```

That command verifies one run by name. In a terminal it looks like this. The first line is the verifier's summary, printed to stderr, so the rows on stdout stay exactly as the reference verifier prints them. When a row fails, it also prints to stderr, under the summary, a `fix (…)` line for each failing row: what to change, and the section of the errors page that explains it. `ONETRACE_QUIET=1` silences these stderr lines too, the summary and the fix lines alike:

<!-- not executed: the run id differs on every run -->
```
$ onetrace-verify --require-artifacts runs/<run id>
record intact: 3 stages, 2 output files checked
[PASS   ] receipts/01-intake.json: receipt format
...
80 pass, 0 fail, 4 not-run  ->  PASS
```

To compare two runs:

<!-- not executed: each run id stands for a folder under runs/ -->
```
onetrace diff runs/<first run id> runs/<second run id>
```

## Switch it off

`ONETRACE_DISABLE=1` makes both decorators pass straight through: the functions run as they did before, and nothing is written.

In a macOS or Linux shell:

<!-- quickstart: run os=posix -->
```
ONETRACE_DISABLE=1 python app.py
```

<!-- quickstart: output -->
```
The warranty covers manufacturing defects for twelve months.
```

In Windows PowerShell:

<!-- quickstart: run os=windows -->
```
$env:ONETRACE_DISABLE = "1"; python app.py; Remove-Item Env:ONETRACE_DISABLE
```

<!-- quickstart: output -->
```
The warranty covers manufacturing defects for twelve months.
```

## Take it out

Delete `import onetrace as ot` from both files, and the three decorator lines. What's left is the code under "Your pipeline", and it runs as it did before. Your run folders stay under `runs/` until you delete them.

<!-- https://oneproof.dev/onetrace/tool-using-agent.md -->

# Recording a tool-using agent

An agent that calls a model, lets the model ask for a tool, runs the tool and calls the model again
is a pipeline whose steps repeat. onetrace records it the same way as any other: `@ot.run` on the
function that does the whole job once, and `@ot.stage` on each step. This page shows the shape with
a stand-in model and an invented order table, so it runs with no network calls and no real data,
then says what to decide for a real one.

## The example

Save this as `agent.py`:

```python
import onetrace as ot

ORDERS = {"A-100": "shipped", "A-101": "waiting for stock"}    # invented; your own system goes here


def stand_in_model(question, tool_result):
    """In place of a hosted model: asks for the order once, then answers from the tool's result."""
    if tool_result is None:
        return {"tool": "order_status", "order_id": "A-101"}
    return {"text": f"Order A-101 is {tool_result}."}


@ot.stage("model", instrument=ot.pkg("stand-in-model", "onetrace", kind="model-call"),
          repeats=True, output_trust="model-generated")
def call_model(question, tool_result=None):
    reply = stand_in_model(question, tool_result)
    ot.constant("model", "stand-in-model")
    return reply


@ot.stage("tool", instrument=ot.pkg("order-status", "onetrace", kind="tool"),
          repeats=True, output_trust="operator-authored")
def run_tool(request):
    return ORDERS.get(request["order_id"], "not found")


@ot.run(stages=["model", "tool"], trust="externally-sourced")
def agent(question):
    reply = call_model(question)
    while "tool" in reply:
        result = run_tool(reply)
        reply = call_model(question, result)
    return reply["text"]


if __name__ == "__main__":
    print(agent("Where is my order A-101?"))
```

Run it and check the record:

```bash
python agent.py
onetrace-verify --require-artifacts runs/<run_id>
```

The run folder holds four stages: `intake` (the question), `model#1`, `tool#1` and `model#2`.
`repeats=True` lets a stage run more than once in a run, numbering the calls.

## How the steps link

Stages link only when the same object is passed on whole (manual, section 16.1). Here:

- `run_tool(reply)` is given the model's whole reply, so `tool#1` records an edge from `model#1`.
  Passing `reply["order_id"]` instead would record the id as an ordinary input, with no edge.
- `call_model(question, result)` is given the tool's whole return value, so `model#2` records an
  edge from `tool#1`.

Keep each step's output whole when you pass it on, and take the part you need inside the next step.

## Trust classes

Each value a stage reads, and each value it returns, has a trust class: where the data came from.
onetrace never guesses one: an unstated class is recorded as `externally-sourced` and listed as
undeclared on that stage. In an agent:

- **What a user typed** is `externally-sourced`, never `operator-authored`. Here `@ot.run`'s
  `trust="externally-sourced"` records the question that way.
- **What a model wrote** is `model-generated`: `output_trust="model-generated"` on the model step.
  That includes the model's request for a tool.
- **What a tool returns** depends on the tool, and is a person's answer to give. A lookup in a
  system you run, as here, is `operator-authored`; a web page, a search result or another party's
  API is `externally-sourced`.

A tool that reads a file or a fetched page records it with its own class, through the running
stage's context:

```python
from onetrace.emit import current_stage

@ot.stage("fetch", instrument=ot.pkg("fetch", "onetrace", kind="tool"), output_trust="externally-sourced")
def fetch(path):
    raw = current_stage().read_external(path, "text/html", name="page", trust_class="externally-sourced")
    return raw.decode("utf-8")
```

`read_external` records the file's digest, its media type and the class you give; the file itself
is not stored. `trust_class="secret"` is refused before anything is digested: keep keys and tokens
out of every stage's arguments (manual, section 5.4).

## Comparing runs with a hosted model

A hosted model can answer differently to the same prompt, and its provider can change it without a
new name. Expect two runs of an agent to differ first at the model step, even when nothing you
control changed.

- **Record what you know about the model** with `ot.constant()`: its name or version, and the
  settings you send, including any you learn only from the reply. A comparison names a setting
  only when it was recorded this way (manual, sections 10.5 and 16.14).
- **Leave `rederivable` unstated** for a hosted model's step: it is recorded as not re-runnable,
  which is what a hosted model is.
- **Test the rest of the agent with a stand-in model**, as above. With the model fixed, two runs
  differ only where your code, your tools or their data differ, and `onetrace diff` names the
  first such step.
- **The console's Compare page looks past a varying model step.** When the first difference is a
  step marked not re-runnable and nothing recorded about it changed except its output, the page
  also names the first later step with a recorded change, marked "First recorded change".

## What this record does not show

- That the agent's answer was correct, or why the model wrote what it wrote.
- Anything a tool did that is not in its recorded inputs and outputs, or any step not decorated.

<!-- https://oneproof.dev/onetrace/manual.md -->

# onetrace: developer manual

This manual takes you from knowing nothing about onetrace to recording, verifying and comparing your own pipeline's runs. Jump to any section from the menu, or from **On this page**.

---

## 1. What onetrace is, in one page

When a pipeline runs, whether it's a RAG system, a document processor or an agent, you usually end up with an output and maybe some logs. Later, when someone asks what exactly produced this answer, and whether anything changed since last week, logs are hard to trust and hard to compare.

onetrace makes the pipeline write a **record** as it runs:

- **one receipt per stage** (a JSON file) that says:
  - what the stage read, identified by content digest (sha256);
  - what it wrote, again by digest;
  - which tool did the work (the *instrument*) and exactly how it was configured;
  - how the stage ended: `ok`, `refused` or `error`;
  - its start and end times;
- **a manifest** that chains the receipts together. Each receipt's digest is recorded in order, ending in a **chain head**. Change, remove or reorder any receipt and the chain no longer matches.

Anyone can then check the record, and compare two runs, with tools that don't trust your code:

| Tool | What it answers |
|---|---|
| `onetrace-verify <run>` | Is this record self-consistent and unbroken? |
| `onetrace diff A B` | Stage by stage, where do two runs differ? |
| `onetrace localize A [B]` | Where did things first go wrong: the first unclean stage in one run, or the first difference between two runs, and its cause? |
| `onetrace reproduce <run>` | If I re-run the recorded stages from their recorded inputs, do I get the same outputs? |

### What onetrace does **not** do (read this before relying on it)

- **It does not say your output is true or correct.** It records what happened and lets others check that record.
- **Records are hash-chained; signing (section 17) and anchoring (section 15) are optional.** The chain proves the record hasn't been altered *relative to its own chain head*. Without a signature, the chain alone does **not** show who wrote it. If you need to prove authorship or when the record existed, keep the chain head somewhere independent, such as a ticket, an email or a database you don't control. To show the record existed by a given time, anchor its chain head after the run closes (section 15).
- **`onetrace-verify` checks your output files under `artifacts/` too, when it can recognise their layout.** If the run was written by this SDK (0.1.x), the verifier finds `artifacts/<stage>/<output>` for each receipt's own outputs and reports a `FAIL` naming any that's missing or changed. If there's no `artifacts/` folder, or its layout isn't one the verifier recognises, it reports one `NOT-RUN` row and checks no files. **The summary line can still say `PASS` in that case**, because `PASS` is about the record. If you need the files checked, run `onetrace-verify --require-artifacts`, which turns that `NOT-RUN` into a `FAIL` (section 8.3).
- **The verifier reports "originality: not-run" on every receipt.** That's expected: a receipt can't show when it was written, so on its own the record establishes consistency, not originality. An anchor (section 15) gets a row of its own beside it, which shows the record existed by the time its source attests; not when the run happened.
- The format is **version 0**. It may still change before version 1.

---

## 2. Install

Requirements: **Python 3.10 or newer**. The console (`onetrace-console`) needs Python 3.12 or newer, so start with 3.12 to use both. Use a virtual environment.

```bash
python -m venv .venv
# Windows PowerShell:  .\.venv\Scripts\Activate.ps1
# macOS / Linux:       source .venv/bin/activate
pip install onetrace
```

This installs two packages:

| Package | Import / command | Role |
|---|---|---|
| `onetrace` | `import onetrace`, command `onetrace` | The SDK: recording, plus the `diff` / `localize` / `reproduce` commands |
| `onetrace-verify` | command `onetrace-verify` | The reference verifier. It uses only the standard library, needs no network, and is installed automatically. |

Check the install:

```bash
python -c "import onetrace; print(onetrace.__version__)"   # 0.4.0
```

To pin exactly what was published, take each file's sha256 from the **Download files** page of the project on PyPI and put it in a requirements file:

```
onetrace==0.4.0         --hash=sha256:<from the PyPI page>
onetrace-verify==0.4.0  --hash=sha256:<from the PyPI page>
```

Then install with `pip install --require-hashes -r requirements.txt`. The hashes can't be printed here, because this manual ships inside one of the files they describe.

Optional extras pin the framework versions the examples were built against:
- `pip install "onetrace[langchain]"`
- `pip install "onetrace[llamaindex]"`
- `pip install "onetrace[langflow]"`

They only install those frameworks. The integration pattern itself is plain Python (section 9).

---

## 3. Five-minute quickstart

Save as `demo.py`:

```python
import json, sys
from pathlib import Path
from onetrace.emit import Instrument, Recorder

QUESTION = "What does the warranty cover?"
CORPUS = [
    {"id": "p1", "text": "The warranty covers manufacturing defects for twelve months."},
    {"id": "p2", "text": "Shipping delays are handled by the logistics partner."},
]

def main(out_dir, run_id):
    out = Path(out_dir); out.mkdir(parents=True, exist_ok=True)
    corpus_path = out / "corpus.json"
    corpus_path.write_text(json.dumps(CORPUS), encoding="utf-8")

    rec = Recorder(out_dir, run_id=run_id,
                   declared_stages=["retrieve", "answer"],
                   manifest=Path(__file__),          # this file's bytes identify the code
                   policy="fail-closed",
                   anchor_reason="not anchored")

    cfg = {"tokenizer": "lower-split", "top_k": 1}   # an int is fine; see section 5.3

    @rec.stage("retrieve", Instrument("word-overlap", "retriever", "1.0.0", cfg))
    def retrieve(ctx):
        corpus = json.loads(ctx.read_external(corpus_path, "application/json",
                                              name="corpus", trust_class="operator-authored"))
        q = set(QUESTION.lower().split())
        best = max(corpus, key=lambda p: len(q & set(p["text"].lower().split())))
        for k, v in cfg.items():
            ctx.constant(k, v)
        ctx.assertion("candidate_count", len(corpus))
        return ctx.write_json("retrieved.json", best)

    @rec.stage("answer", Instrument("extractive", "answerer", "1.0.0", {"method": "extractive"}))
    def answer(ctx, retrieved_artifact):
        hit = ctx.read_json(retrieved_artifact)
        ctx.constant("method", "extractive")
        ctx.assertion("source", hit["id"])
        return ctx.write_json("answer.json", {"answer": hit["text"], "cited": hit["id"]})

    answer(retrieve())
    rec.close()

if __name__ == "__main__":
    main(sys.argv[1], sys.argv[2])
```

Run it twice, verify one run, and compare the two:

```bash
python demo.py run_a run-a
python demo.py run_b run-b
onetrace-verify run_a          # ... "56 pass, 0 fail, 2 not-run  ->  PASS"
onetrace diff run_a run_b      # "diff: identical", exit 0
```

---

## 4. Core concepts

| Term | Meaning |
|---|---|
| **Run** | One execution of your pipeline, written to one folder. A run folder is **write-once**: onetrace refuses to write into a folder that already holds a run. |
| **Stage** | One named step. You declare the stage names, in order, when the run starts, and the stages must emit in that order. |
| **Receipt** | The JSON record of one stage: `receipts/NN-<stage>.json`. |
| **Manifest** | `MANIFEST.json`: the ordered chain of receipt digests and the **chain head**. It is rewritten after every receipt, and is the run's "last word". |
| **Instrument** | The tool a stage used: id, kind, a pinned version, and its configuration. The configuration is digested, so a config change is always visible. |
| **Artifact** | A file a stage wrote through onetrace. It's stored under `artifacts/NN-<stage>/` and its digest goes in the receipt; `onetrace-verify` checks it there (section 8). |
| **Constant / assertion** | Constants are the settings the stage ran under. Assertions are facts the stage states about its work (a count, a chosen id). Each takes a string, a number or a boolean, recorded as a string (section 5.6). |
| **Trust class** | Where data came from: `operator-authored`, `model-generated` or `externally-sourced`. `secret` is refused, so secrets never enter a record. |
| **Policy** | `fail-closed` or `fail-open`: what happens when a receipt can't be written or verification fails (section 5.8). |
| **Coverage** | Which declared stages have emitted. `complete` only when every declared stage has emitted and no boundary or gap stands. |
| **Boundary / gap** | An honest statement that the record doesn't see past a point (a boundary), or that a receipt is missing (a gap). Either makes coverage `incomplete`. |

---

## 5. Integrating onetrace into your code, step by step

### 5.1 Create one `Recorder` per run

```python
from pathlib import Path
from onetrace.emit import Recorder

rec = Recorder(
    "runs/run-0001",              # out_dir: a NEW folder for this run
    declared_stages=["load", "chunk", "embed", "retrieve", "answer"],
    manifest=Path(__file__),           # the file whose bytes identify your pipeline code
    policy="fail-closed",              # or "fail-open"
    anchor_reason="not anchored",      # optional: why the receipts are unanchored (section 15 anchors a closed run)
    run_id=None,                       # optional; generated if omitted
)
```

| Argument | Required | Notes |
|---|---|---|
| `out_dir` | yes | Created if missing. It must not already contain a run. One process per folder: a lock file enforces this. |
| `declared_stages` | yes | A non-empty list of distinct names, in execution order. |
| `manifest` | yes | A path to a file, usually your pipeline's main module. Its sha256 is recorded as every instrument's `manifest_digest`. `reproduce` uses it to confirm it's re-running the same code. |
| `policy` | yes | `"fail-closed"` or `"fail-open"`. Anything else is refused. |
| `run_id` | no | Your id. If omitted, one is generated: a UTC timestamp plus 8 random hex characters, which sorts chronologically. |
| `anchor_reason` | no | A free-text note, recorded with `anchor.state = "unanchored"`. |
| `instruments` | no | Declare instruments up front, keyed by stage name (5.3). |
| `declared_edges` / `topology` | no | For branching pipelines (section 6). |
| `boundaries` | no | Boundaries known at start (5.9). |
| `verify_pin` | no | Run the verifier automatically on `close()` (5.10). |
| `max_run_bytes` | no | A hard size limit for the run (5.11). |

### 5.2 Wrap each step as a stage

Use the decorator. onetrace passes a **context** (`ctx`) as the first argument. Your other arguments pass through unchanged, and your return value comes back to the caller.

```python
from onetrace.emit import Instrument

@rec.stage("chunk", Instrument("recursive-splitter", "chunker", "2.3.1",
                               {"chunk_size": 900, "overlap": 0}))
def chunk(ctx, doc_artifact):
    text = ctx.read(doc_artifact).decode("utf-8")
    ctx.constant("chunk_size", 900)
    ctx.constant("overlap", 0)
    chunks = split(text, 900)
    ctx.assertion("chunk_count", len(chunks))
    return ctx.write_json("chunks.json", chunks)

chunks_artifact = chunk(doc_artifact)
```

**Stages must run in the declared order.** Calling `answer` before `retrieve` raises `EmissionRefused: stage 'answer' is not the next declared stage (expected retrieve)`.

The same stage can be written without a decorator: `rec.run_stage("chunk", instrument, fn, *args)`.

### 5.3 Describe the instrument honestly

```python
Instrument(id, kind, version, config, rederivable="true", rederivable_note=None)
```

- `id`: your name for the tool, for example `"gpt-4o-mini"` or `"recursive-splitter"`.
- `kind`: its role, for example `"retriever"`, `"llm"` or `"chunker"`.
- `version`: **must pin an exact build.** `""`, `latest`, `current`, `head`, `main` and `master` are refused. Use the library version (`importlib.metadata.version("langchain-core")`) or a model snapshot id.
- `config`: a dict of the settings that affect the output. A **string** is always fine. A Python **`int`** is recorded as its own decimal string, a **`bool`** as `"true"` or `"false"`, and a **`float`** in RFC 8785's number form (the shortest decimal that reads back as the same float), so `3` and `"3"`, or `0.7` and `"0.7"`, produce the same `config_digest` (rule M, section 16.16). NaN and Infinity have no such form and are refused, naming where they were given. Nested dicts and lists follow the same rule at every level.
- `rederivable="false"`: set this when re-running can't reproduce the output, for example a sampled LLM call. Explain why in `rederivable_note`.

If you don't want to repeat instruments at every call site, declare them once:

```python
rec = Recorder(..., instruments={"chunk": Instrument(...), "answer": Instrument(...)})

@rec.stage("chunk")          # no instrument here; it comes from the declaration
def chunk(ctx, ...): ...
```

A run that declares instruments refuses a stage that brings its own, or one that has none declared.

### 5.4 Inputs: everything a stage reads goes through `ctx`

| Call | Use it for | Returns |
|---|---|---|
| `ctx.read_external(path, media_type, name=None, trust_class="externally-sourced")` | a file from outside the run (a document, a config, a prompt file) | `bytes` |
| `ctx.read_memory(data, media_type, name, trust_class="externally-sourced")` | an input already in memory, with no path on disk (a request body, a user query). `name` is required — there's no path to default it from. Digested, never stored, exactly like `read_external`. | `bytes` (the same `data`) |
| `ctx.reference(path, media_type, name=None, trust_class="externally-sourced", digest=None)` | a **large** file you don't need in memory. It's streamed and digested in bounded memory. If you pass `digest=`, a mismatch is refused. | an `Artifact` |
| `ctx.read(artifact)` | an artifact written by an earlier stage | `bytes` (a digest mismatch is refused) |
| `ctx.read_json(artifact)` | the same, parsed as JSON | the object |
| `ctx.read_receipt(node)` | depend on an earlier stage's **receipt**; this is what records an edge in a branching pipeline | the receipt |

`trust_class="secret"` is **refused** on every input path, so a stage can't put a secret's digest into a record. Don't route API keys or credentials through `ctx`.

A value passed to a decorated stage as an argument is recorded as its digest, never its text. A digest of a value that can be guessed (a short PIN, an email address, a key from a small set) can be matched by trying the guesses, so a digest is not a secret's protection: keep such values out of stage arguments, or declare them `secret`.

Reads that bypass `ctx`, such as a direct `open()` or an HTTP call, aren't recorded. That's allowed, but the receipt won't mention them. Put anything that affects the output through `ctx`.

### 5.5 Outputs

```python
art = ctx.write("report.pdf", pdf_bytes, "application/pdf", trust_class="operator-authored")
art = ctx.write_json("answer.json", obj, trust_class="model-generated")
```

- Files are stored under `artifacts/NN-<stage>/<name>` and digested over the stored bytes. `onetrace-verify` checks each one against its receipt (section 8).
- `write_json` writes canonical JSON, so **strings only, no JSON numbers** — this is unaffected by section 5.3's int-acceptance, which is metadata (config/constants/assertions), not your own stored data. Silently rewriting a caller's data is worse than refusing it, so a number here is still refused outright; put numbers in as strings: `"0.87"`.
- `write_json` needs a JSON **object** at the top level. Wrap a list: `ctx.write_json("docs.json", {"docs": docs})`.
- Use `trust_class="model-generated"` for anything an LLM produced. Readers rely on it.
- **State the class on every write.** A `ctx.write` or `write_json` with no `trust_class` records
  `operator-authored` and lists nothing as undeclared, whatever the content is. A decorated
  stage's unstated output is recorded as `externally-sourced` and listed (16.2); a hand-written
  write is not, in this version. `externally-sourced` is the class for content that arrived from
  outside: documents, passages, a prompt that quotes them.
- The name is one file name, stored in the stage's own folder: no `/` or `\`, not `.` or `..`,
  no drive, and nothing Windows would store under another name (a reserved device name such as
  `CON`, `< > : " | ? *`, a control character, a trailing space or dot, over 255 bytes). Any
  other name is refused before anything is written.
- Pass the returned `Artifact` to the next stage and read it there with `ctx.read` or `ctx.read_json`. That's how the chain records where each stage's inputs came from.
- Once the stage has ended, `artifact.path` is the file's location relative to the run folder, for example `artifacts/01-retrieve/retrieved.json`.

### 5.6 Constants and assertions

```python
ctx.constant("top_k", 4)                   # a setting the stage ran under
ctx.assertion("hits", 4)                   # a fact the stage states about its work
```

- Each takes a **string**, a number or a boolean, recorded as a string by the same rule as `Instrument.config` (section 5.3): `512` as `"512"`, `False` as `"false"`, `0.25` as `"0.25"`. NaN and Infinity are refused.
- **A stage that makes assertions must declare at least one constant.** Otherwise you'll get `EmissionRefused: ... assertions [...] declare no constant`. The rule exists so that no stated fact can depend on an unstated setting.

### 5.7 How a stage ends

| Your function… | Receipt `outcome` | What your caller sees |
|---|---|---|
| returns normally | `{"class": "ok"}` | the return value |
| raises `Refusal(reason, detail)` | `{"class": "refused", "reason": ..., "detail": ...}` | `None`; the run continues |
| raises any other exception | `{"class": "error", "status": <type>, "body": <message>, "origin": <instrument id>}` | the exception, re-raised after the receipt is written |

Use `Refusal` when your stage deliberately declines, for example a policy check fails or the input is empty:

```python
from onetrace.emit import Refusal

@rec.stage("answer", instr)
def answer(ctx, hits_artifact):
    hits = ctx.read_json(hits_artifact)
    if not hits:
        raise Refusal("no supporting passages", "retriever returned 0 hits")
    ...
```

### 5.8 `fail-closed` or `fail-open`

This governs what happens if a **receipt can't be written**, or, with `verify_pin`, if **verification fails**:

- **`fail-closed`:** the stage's result is discarded and `EmissionRefused` is raised. A failed verification withdraws the manifest. Use this where a result without a record must not be used, as in regulated, audited or customer-facing flows.
- **`fail-open`:** the result stands. The missing receipt is recorded as a **gap**, and coverage becomes `incomplete`. Use this where availability matters more and a visible hole in the record is acceptable.

Either way, the record never pretends: a missing receipt is written down as missing.

### 5.9 Boundaries and gaps

A **boundary** says the record can't see past a point, for example a stage that calls a third-party system you can't instrument:

```python
rec = Recorder(..., boundaries=[{"name": "answer", "kind": "external service",
                                 "note": "vendor API; its implementation is not visible"}])
# or during the run:
rec.boundary("answer", "external service", "vendor API; its implementation is not visible")
```

A **gap** records a receipt you know you failed to write:

```python
rec.gap("embed", "embedding worker crashed before emitting")
```

Both must name a **declared** stage, and both make coverage `incomplete`. They exist so that a record can be honest about what it doesn't show.

### 5.10 Closing the run, and verifying automatically

**Always call `rec.close()`** at the end, in a `finally:` block if stages may raise. It writes the final manifest, including any gaps or boundaries declared after the last receipt, and runs the verify hook if you configured one:

```python
from onetrace.verify_pin import VerifyPin

rec = Recorder(..., policy="fail-closed", verify_pin=VerifyPin())
try:
    ...stages...
finally:
    rec.close()     # writes MANIFEST.json, then runs onetrace-verify and writes VERIFIER_CALL.json
```

- The verifier's result goes **beside** the record in `VERIFIER_CALL.json` (`result`, `exit`, `rows`, and the sha256 of the verifier files that ran). It is never written into the record itself.
- `VERIFIER_CALL.json` uses two spellings of the third outcome, on purpose. Its own `result` uses the SDK's words and exit codes: `PASS` 0, `FAIL` 1, `NOT RUN` 4. Its `rows` are `onetrace-verify`'s own, which spell it `NOT-RUN`. They mean the same thing; each spelling is its own tool's contract. An `onetrace-verify` exit 2 (`NOT VERIFIED`, or refused, section 10.5) is recorded there as `NOT RUN`, exit 4.
- `VerifyPin(expect_sums="<sha256>")` pins the exact verifier build. A different verifier is not run, and the mismatch is reported.
- `VerifyPin(per_receipt=True)` also verifies after every receipt.
- Under `fail-closed`, a verification failure raises and the run is not declared complete.

### 5.11 Limits, concurrency and async

- **Size bound:** `Recorder(..., max_run_bytes=500_000_000)`. A write that would exceed it raises `StoreExhausted`. That derives from `BaseException`, like `KeyboardInterrupt`, so an ordinary `except Exception` doesn't swallow it. A receipt naming the limit is written for the stage in progress when the bound leaves room for it. `max_run_bytes` itself is refused at construction if it is below the smallest a real receipt for your own declared stages could ever be — measured fresh from your run's own configuration each time, not a fixed number — so a bound too small to hold even that receipt is never accepted in the first place, rather than left to fail later with no receipt and no gap to explain why. A write whose own output already reached disk before the bound stopped it is cleaned up: the bound is a controlled refusal, not a crash, so no orphaned `*.tmp` file is left under `artifacts/`.
- **One process per run folder.** The `.lock` file enforces it. Threads and asyncio tasks inside that process are fine, because emission is serialised inside the recorder.
- **Async stages:**

```python
async with rec.stage("answer", instr) as ctx:
    hits = ctx.read_json(hits_artifact)
    ctx.constant("model", "m-1")
    result = await call_model(hits)
    ctx.write_json("answer.json", result, trust_class="model-generated")
```

---

## 6. Branching pipelines (DAGs)

If your pipeline fans out and back in, declare the approved edges between stage names, and run each instance with `run_node`:

```python
rec = Recorder("runs/dag-1", declared_stages=["load", "normalise", "merge"],
               manifest=Path(__file__), policy="fail-closed",
               declared_edges=[{"from": "load", "to": "normalise"},
                               {"from": "normalise", "to": "merge"}])

def normalise(ctx, key):
    ctx.read_receipt("load")                      # records the edge load -> normalise#<key>
    return ctx.write_json(f"{key}.json", {"k": key})

def merge(ctx):
    ctx.read_receipt("normalise#a")               # instances are named stage#instance
    ctx.read_receipt("normalise#b")
    return ctx.write_json("merged.json", {"ok": "yes"})

rec.run_node("load", None, instr_load, load)
rec.run_node("normalise", "a", instr_norm, normalise, "a")
rec.run_node("normalise", "b", instr_norm, normalise, "b")
rec.run_node("merge", None, instr_merge, merge)
rec.close()
```

- Edges are **observed** from `read_receipt` calls and written into the manifest. Each one is checked against `declared_edges` as it happens.
- **With the decorators, the plan is checked before a stage runs** (0.3.0). A stage called with an earlier stage's output along a path `declared_edges` does not hold does not run: its receipt is written with outcome `refused`, the reason naming the path, and the run ends with `EmissionRefused`. Since 0.4.0 (format 0.3) the receipt keeps the link it would have read under `refused.attempted`, not under `inputs`, so the run's edges stay within the plan: `onetrace-verify`'s plan row does not fail, and its row "step held before running" names each attempted path (`route -> tool_b`). A held step tells you the plan stopped something; an off-plan edge in the record (a `Recorder` stage that ran) still fails the plan row. The verbs read a run with a held step as refused, exit 3, "a step was held before running (route -> tool_b)": the run stopped there. Only a refused receipt can carry `refused.attempted`; any other does not verify. Before 0.4.0 the attempted link was kept under `inputs`, so the plan row failed and the run never read `PASS`; a verifier from before 0.4 reads a 0.3 record with a held step as PASS too, without naming the path. A `Recorder` stage that calls `read_receipt` inside its own body is checked when that receipt is written, as before.
- A run that declares `declared_edges` but records no edge (a framework that hands each step a fresh copy of its state, say) gets the plan row `NOT-RUN`, `no edges observed`: the plan was declared and nothing was checked against it.
- A bare stage name that matches several instances is refused as ambiguous. Name the instance.
- On a DAG, `onetrace localize <run>` reports the **set** of earliest unclean nodes, sorted, never a single "first" node.
- Linear and DAG runs are **not comparable** with each other: `diff` reports `not comparable`, with exit code 2.
- **Concurrent stages** (two nodes racing for the same emission lock) are supported; onetrace-verify's own artifact check (section 8) accounts for the race correctly as of 0.1.1 (section 14).

---

## 7. What a run folder contains

```
runs/run-a/
├── .lock                       one-process guard (leave it)
├── MANIFEST.json               the chain: receipt digests in order, and chain_head
├── receipts/
│   ├── 01-retrieve.json        one canonical JSON receipt per stage
│   └── 02-answer.json
├── artifacts/
│   ├── 01-retrieve/retrieved.json
│   └── 02-answer/answer.json
└── VERIFIER_CALL.json          only if verify_pin was set
```

The files you pass to `read_external` stay where they are. The quickstart writes `corpus.json` into the run folder only for convenience.

**What a run folder stores.** Outputs are kept: a run folder holds each stage's outputs, which can include documents, prompts and answers. Inputs are fingerprinted, not stored: a receipt records an input's digest, length and trust class, never its bytes. So treat run folders like logs that may contain sensitive data.

A receipt, abridged:

```json
{"format": "stage-receipt/0.2",
 "stage": {"index": "2", "name": "answer"},
 "instrument": {"id": "extractive", "kind": "answerer", "version": "1.0.0",
                "config_digest": "sha256:…", "manifest_digest": "sha256:…", "rederivable": "true"},
 "inputs":  [{"name": "retrieved.json", "digest": "sha256:…", "trust_class": "operator-authored", …}],
 "outputs": [{"name": "answer.json", "digest": "sha256:…", "bytes": "86", …}],
 "assertions": {"source": "p1", "constants": {"method": "extractive"}},
 "outcome": {"class": "ok"},
 "coverage": {"completeness": "complete", "declared_stages": […], "emitting_stages": […], "boundaries": []},
 "emission": {"policy": "fail-closed", "gaps": []},
 "anchor": {"state": "unanchored", "reason": "not anchored"},
 "time": {"started": "…Z", "ended": "…Z"},
 "prev": "sha256:<previous receipt>", …}
```

---

## 8. Verifying a run

### 8.1 `onetrace-verify`

```bash
onetrace-verify runs/run-a            # a run folder, or a path to MANIFEST.json
onetrace-verify --help                # usage
onetrace-verify --version             # 0.4.0
onetrace-verify --require-artifacts runs/run-a   # also FAIL if the output files can't be checked
onetrace-verify --json runs/run-a     # machine-readable rows and a summary, for a CI gate
onetrace-verify --require-signature runs/run-a                   # FAIL unless a valid signature is beside the run
onetrace-verify --require-trusted --trust trust.json runs/run-a  # FAIL unless it is by a key you trust
onetrace-verify --require-complete runs/run-a                    # FAIL unless every declared stage emitted
onetrace-verify --headers headers.json runs/run-a               # check OpenTimestamps anchors (15.5)
onetrace-verify --tsa-cert tsa.pem --tsa-root root.pem runs/run-a   # check RFC 3161 anchors (15.4)
```

The three `--require-` gates (0.3.0) are for CI; without them nothing changes. Each adds one row after
the records beside the chain, `PASS` or `FAIL`, so a `FAIL` gives exit 1:
- `--require-signature`: `signature required` passes when a valid signature over the chain head is in
  `signatures/`, by any key. A run never signed, or whose `signatures/` was removed, fails.
- `--require-trusted`: also `trusted signature required`, which passes only when a valid signature is by
  a key in `--trust` (it needs `--trust`; without it, a usage error, exit 2). A run re-signed with
  another key fails.
- `--require-complete`: `run complete` passes when the run's last receipt says it is complete: every
  declared stage emitted a receipt. A run that stopped early, or a branch that skipped a declared
  stage, fails, naming the stages that never emitted. A stage that errored did emit, so this gate reads
  coverage, not outcomes. **Optional stages** (since 0.4.0, format 0.3): a stage declared
  `@ot.stage(..., optional=True)`, or listed as `ot.optional("rerank")` in `@ot.run(stages=[...])`,
  may be skipped. The run's receipts name it under `coverage.optional_stages`, and a run that skipped
  one still says `incomplete`, so a verifier from before 0.4 reads it as it always did (its
  `--require-complete` fails such a run: stricter, never looser). onetrace-verify 0.4's
  `run complete` passes when only optional stages didn't emit, and names them ("optional, not run:
  rerank"); a required stage that didn't still fails, by name. A comparison of a run that ran it with
  one that skipped it reads `present in A only (optional)`: a difference in shape, never "same".
  Optional belongs to the stage's declaration: a stage name declared optional in one place and
  required in another is refused. In a run written with `Recorder`, pass
  `optional_stages=[...]`; its stages still run in declared order, and only an optional one may be
  skipped.

An option the verifier doesn't know, a typo such as `--require-signatures` or one from a later
version, is a usage error, exit 2, from 0.3.0. Versions before 0.3.0 ignore options they don't know
and have no gates, so a run would pass them unchecked: in CI, pin `onetrace-verify>=0.3.0` or check
`onetrace-verify --version` first.

`--json` replaces the row-by-row text with one JSON document: `{"rows": [{"result", "name",
"detail"}, ...], "summary": {"pass", "fail", "not_run", "result", "exit"}}`, using the same words
the text rows use. The exit code is the same either way: `--json` changes only how the result
is printed, not what is checked.

It prints one row per check: each receipt's own format, canonical bytes, required members, digests matching the manifest, `prev` links, stage order, coverage, the chain head, and — when it can recognise the run's own `artifacts/<stage>/<output>` layout — every output file against the digest its receipt recorded (section 8.3). It ends with a summary:

```
56 pass, 0 fail, 2 not-run  ->  PASS
```

| Exit | Meaning |
|---|---|
| 0 | PASS: the record is consistent and unbroken |
| 1 | FAIL: at least one check failed, and the `[FAIL]` rows say which |
| 2 | NOT VERIFIED: a receipt's own format is one this verifier doesn't implement, and nothing failed (section 10.5). Also exit 2: refused, when the manifest can't be read or its chain format is unknown (refused by name, never guessed), or no manifest found. |

In human mode it also writes to stderr, before the rows: one summary line drawn only from the rows it computed, which reads `record intact: 2 stages, 2 output files checked`, `record NOT intact: <the first failing row>` (with the number of failing rows when there are several), or `record not verified: <reason>`. After it comes one line for each `FAIL` row, and none for a `NOT-RUN` or `PASS` row: `fix (<row>): <what to do>; see https://oneproof.dev/onetrace/errors.md#<family>`, where the family is the errors page's section for that kind of check: `verify-receipt`, `verify-chain`, `verify-artifacts`, `verify-anchor`, `verify-signature` or `verify-plan`. These lines change nothing else: the rows and the summary on stdout, the exit code, and `--json` (which writes nothing to stderr) are as above. `ONETRACE_QUIET=1` silences them (section 16.18).

`not-run` rows for a receipt's `originality` are normal (section 1). A run with anchors also gets one row per anchor in `anchors/` (15.3): a `FAIL` fails the run like any other, and a `NOT-RUN` ("needs block headers", say) leaves the result as it would be without it, which a line after the summary says. A run with an anchor row also ends with one more line, what an anchor shows and doesn't: "anchors show this record existed by the time the source attests; not when the run happened, that its outputs were correct, or who wrote it." Neither line is printed with `--json`; the first line's count is there, as `summary.records_not_run`. (`onetrace-verify` spells the outcome `NOT-RUN`; the SDK's own verifier, `python -m onetrace.sdk_verify`, and `VERIFIER_CALL.json`'s own `result` spell it `NOT RUN`, section 5.10.) A `not-run` row can also name `run: artifacts`, when there's no `artifacts/` directory at all, or when its layout isn't one this verifier recognises (section 8.3). A `not-run` row never turns the summary into `FAIL`, so read the rows, not just the last line, or use `--require-artifacts`. `python -m onetrace.sdk_verify` checks the output files the same way, and gives the same `run: artifacts` row (`NOT RUN`) when there are none to check; it has no `--require-artifacts`. A `receipt format` `not-run` row is different: that receipt's content was not checked, and it turns the summary into `NOT VERIFIED`, exit 2.

A missing, unreadable or malformed file the manifest names — a removed receipt, a truncated write, a directory where a file should be — gives a `[FAIL]` row naming it, never a Python traceback (fixed in 0.1.1; section 14).

### 8.2 What tampering looks like

Edit a receipt and the verifier fails it, and the chain head:

```
[FAIL   ] receipts/02-answer.json: digest matches the manifest
[FAIL   ] manifest: chain head matches the last receipt
54 pass, 2 fail, 2 not-run  ->  FAIL
```

### 8.3 Output files are checked automatically

As of 0.1.1, `onetrace-verify` reads `artifacts/` itself — you no longer need a separate script. For every receipt's own recorded outputs, it looks for `artifacts/<receipt stem>/<output name>` (the layout this SDK's own `ctx.write`/`write_json` produce):

- **A run with a recognised `artifacts/` layout:** every recorded output must be present with a matching digest, or it's a `[FAIL]` row naming the file. A receipt whose own format this verifier doesn't implement is skipped: its outputs are content it can't read.
- **A run with no `artifacts/` at all, or one whose layout this verifier doesn't recognise** (an older run, or one built by hand rather than through this SDK): one `NOT-RUN` row, `run: artifacts`, naming which case it is. No file is checked. The summary can still read `PASS`, because the record itself checks out.
- **With `--require-artifacts`**, that `NOT-RUN` row becomes a `FAIL` and the exit code is 1. Use it whenever the files matter, for example when someone hands you a run folder, and in CI. Otherwise, deleting the whole `artifacts/` folder would leave the summary at `PASS`.
- **A run that declares it keeps no output files** (since 0.4.0, format 0.3): `@ot.run(..., keep_outputs=False)`, or `Recorder(..., keep_outputs=False)`, records every output's digest and size as before and removes the files when the run closes; its manifest says `outputs_kept: "none"`. onetrace-verify 0.4 reads the declaration: `run: artifacts` is NOT-RUN, "the run declares it keeps no output files", never PASS, and `--require-artifacts` FAILs on it, naming the declaration. A file found beside such a run is not read, so nothing left behind is taken for a checked file. The verbs compare its outputs by digest; what needs the files (a text diff, masking) says the output was not kept.
- **The flag requires the `artifacts/` folder itself.** A run that wrote no output file at all (its only stage errored, say) has no `artifacts/` and fails under the flag; the same error after a stage that wrote an output leaves the folder, whose files are checked, and passes. To gate on whether a run finished, add `--require-complete`.

```bash
onetrace-verify runs/run-a
```

```
...
[FAIL   ] receipts/02-answer.json: output artifact 'answer.json' -- missing from artifacts/
...
55 pass, 1 fail, 2 not-run  ->  FAIL
```

Editing a file under `artifacts/` (or deleting one) now fails the verifier directly — it no longer takes a receipt edit to notice.

---

## 9. Framework integrations (LangChain, LlamaIndex, Langflow)

There is no magic callback. Integration means **wrapping each logical step of your chain as a stage**, so that everything it reads and writes goes through `ctx`. The published source distribution includes complete, runnable examples under `examples/`: `langchain_adapter`, `llamaindex_adapter`, `langflow_adapter`, and `plain_python_control`, a framework-free baseline.

**LangChain, since 0.4.0: `onetrace.langchain`** (`pip install "onetrace[langchain]"`). Importing it registers
encoders for LangChain's own types, so a stage can take or return them and two runs of the same chain
record the same bytes: a message records every field but its `id`, `response_metadata` and
`usage_metadata`, which differ on every call (so its role, tool calls with their ids, invalid tool
calls, a tool message's status and artifact, and `additional_kwargs` are all kept); a prompt value
records its messages; a document its page content and metadata, without its `id`; an LLM result each
generation's text, message and `generation_info` (a finish reason, say), without token counts. A field
not named here as left out is kept. A hosted model gives each tool call a new id on every call, so
with one the model stage is the first difference, as it is for any hosted model. `wrap(runnable, "name", ...)`
makes one Runnable a stage (every `ot.stage` option applies) and returns a Runnable, so an LCEL
chain records one node per step, each linked to the one before it:

```python
import onetrace as ot
from onetrace.langchain import wrap

chain = (wrap(prompt, "prompt", output_trust="operator-authored")
         | wrap(llm, "llm", output_trust="model-generated", rederivable=False)
         | wrap(StrOutputParser(), "parse", output_trust="model-generated"))

@ot.run(stages=["prompt", "llm", "parse"])
def answer(q):
    return chain.invoke({"q": q})
```

**LangGraph, since 0.4.0: `from onetrace.langgraph import stage`**, in place of `ot.stage` on each
node, records which node wrote each key of the state a node is given (section 16, *Lineage through a
framework's state*).

The pattern, using LangChain:

```python
import importlib.metadata as md
from langchain_text_splitters import RecursiveCharacterTextSplitter
from onetrace.emit import Instrument

LC = md.version("langchain-core")          # pin the real installed version

@rec.stage("split", Instrument("RecursiveCharacterTextSplitter", "splitter",
                               md.version("langchain-text-splitters"),
                               {"chunk_size": 900, "chunk_overlap": 0}))
def split(ctx, cleaned_artifact):
    text = ctx.read(cleaned_artifact).decode("utf-8")
    ctx.constant("chunk_size", 900); ctx.constant("chunk_overlap", 0)
    chunks = RecursiveCharacterTextSplitter(chunk_size=900, chunk_overlap=0).split_text(text)
    ctx.assertion("chunk_count", len(chunks))
    return ctx.write_json("chunks.json", chunks)

@rec.stage("answer", Instrument("my-llm", "llm", "model-snapshot-01",
                                {"temperature": 0}, rederivable="false",
                                rederivable_note="hosted model; sampling not reproducible"))
def answer(ctx, prompt_artifact):
    prompt = ctx.read(prompt_artifact).decode("utf-8")
    ctx.constant("temperature", 0)
    reply = llm.invoke(prompt)
    ctx.assertion("reply_chars", len(reply))
    return ctx.write("answer.txt", reply.encode("utf-8"), "text/plain", trust_class="model-generated")
```

Guidelines:

- **One stage per step you'd want to point at** when something goes wrong: load, convert, clean, split, embed, index, retrieve, build the prompt, answer.
- **Version and configuration come from the real objects,** read at run time, never typed in by hand.
- **Mark every LLM output `model-generated`,** and every hosted-model stage `rederivable="false"`.
- **Vector stores and embeddings are usually in memory.** Write what matters as artifacts (ids, scores, the prompt) so that the next stage reads it through `ctx`.

---

## 10. Comparing runs

### 10.1 `diff`: a stage-by-stage ladder

```bash
onetrace diff runs/monday runs/tuesday --out reports/diff
```

```
stage         baseline            candidate           verdict
retrieve      51bd312473c4        51bd312473c4        same
answer        35a281ec6835        9f02c1d7a4be        FIRST DIFFERENCE
```

Each stage gets exactly one of five verdicts, always about the stage's **output**:

| Verdict | Meaning |
|---|---|
| `same` | the output digests match |
| `FIRST DIFFERENCE` | the first stage, in order, whose output differs |
| `downstream` | it differs after the first difference, so the divergence propagated |
| `reconverged` | it matches again after diverging |
| `COULD NOT CHECK` | it can't be evaluated (unknown format, a boundary, an unreadable file). **This is not a pass.** |

A change in an instrument's version or config never changes a verdict by itself. It's shown as an annotation, so a `same` never hides that *how* the output was produced changed.

**Different inputs are said first** (0.3.1). When the two runs were given different recorded inputs, the line after the verdict says so and names them: `inputs differ: the runs were given different recorded inputs: q (the intake stage's inputs)`, then that the stages below are compared across different inputs, with the `onetrace match` command that finds a run on the same input (see [runs on the same input](#runs-on-the-same-input-onetrace-match)). A run's recorded inputs, here, are the arguments it was called with: its intake stage's inputs (section 16.3), the job it was given, such as a ticket or a question. An input a stage reads on its own, such as a corpus or a config file, is not one of them: a release changes those, and the ladder compares them stage by stage. A run with no intake stage (a run function without parameters, or a run not recorded with `@ot.run`) records no arguments, and nothing is said. They are compared by digest. The JSON report carries them under `inputs_differ`. The verdicts and the exit code are unchanged, but an `identical` or `identical after masking` verdict on different inputs says so on the verdict line itself, `localize: identical after masking, on different inputs (q)`, and the JSON carries `"on_different_inputs": "true"`: a mask that names the input's own kind of value (a path given as the argument, say) can make two runs on different inputs read the same, and a reader of the verdict alone should not take them for one run.

**The outputs that changed at the end** (0.3.1). When two runs diverge, the report lists each stage nothing else reads from whose output differs, with where its change starts (its origins; see [one output's path](#one-outputs-path-localize---for)), under `end_outputs` in the JSON. The text prints the list as `outputs that changed at the end:` when it says more than the first difference does: more than one such stage, or one whose change starts somewhere else. On a straight line of stages there is one, and it isn't printed. When the origins are more than one stage, the report ranks them (`origins_ranked`): each stage, with how many of those outputs its change reaches. A stage none of whose recorded inputs differs at some origin ("nothing it read differs") ranks above one that was only ever given an input that differs and that no recorded link traces to a stage ("an input it was given differs"), since the second may only be passing on a change from outside the record; then most outputs first. The list and the ranking never change a verdict or the exit code.

#### Release noise: `--mask FILE`

A release changes things nobody needs to look at: a version string, a generated time, the folder a run happened in. `--mask FILE` (0.3.1, on `diff` and two-run `localize`) names them once, so the first difference is one that matters. It is opt-in: with no mask, nothing below applies.

```json
{"format": "onetrace-mask/0.1",
 "settings": ["instrument.version", {"stage": "load", "setting": "assertions.constants.app_version"}],
 "values":   ["timestamp", "uuid", "run-id", "absolute-path"],
 "fields":   [{"stage": "answer", "output": "return.json", "pointer": "/meta/generated_at"}]}
```

- **`settings`:** names as the annotations name them (`instrument.version`, `assertions.constants.<key>`), for every stage or, as an object, for one. A masked setting is left out of the annotations and the cause, and listed as `masked at stage …: … (not counted as a change)`.
- **`values`:** from a fixed set only, no free-form patterns: ISO 8601 timestamps, UUIDs, onetrace run ids (both runs' own, and any in run-id form), absolute paths. For paths, each run's own folder is replaced whole first; then every other absolute path (a POSIX path of two names or more, a Windows path from a drive letter) keeps its last name and only the folders above it are replaced, so `/etc/app/prod.yaml` against `/etc/app/staging.yaml`, or the routes `/v1/refund/approve` against `/v1/refund/deny`, still differ. A path whose folders alone differ between the runs (`/opt/models/v1/model.bin` against `/opt/models/v2/model.bin`) is masked, and the report lists it under the mask line, so you can check it is the same file (a path under either run's own folder is masked whole and not listed; paths that share a last name are each listed, paired by the folders they share from the root, and a path with no partner reads `path | -`; the JSON carries up to 100 such paths per output and `folders_masked_count`, the number of all of them, and the text names the rest by count). In text that reads as JSON (one value, or any line that is one, whatever its media type; a string that is itself JSON text is read one level down), values are looked for inside each string as the string holds them, so a path after an escaped newline, or a Windows path with escaped backslashes, is found; a string the mask changes nothing in keeps its bytes. Each is replaced in the output's text with a placeholder, and nothing else about the bytes changes: a change in spacing or key order still differs.
- **`fields`:** JSON pointers inside one stage's output (`output` left out: every JSON output of that stage). An output with fields named is read as JSON, the fields replaced, and compared as JSON, so its key order and spacing are not counted; the row says `as JSON`. A pointer that reaches no value in either run (a key neither has, or one that runs into a string or number before its end, such as a field inside JSON text held in a string) replaces nothing, and the mask line lists it under "masked fields no value reaches in either run".

Receipts hold digests, so a masked comparison reads both runs' kept output files, by size and digest. A stage whose outputs are the same once masked gets the verdict they would have had if equal, and says so: `same  (same after masking (2 values): out.json)`. Never a plain `same`. A mask that replaced nothing hides nothing. An output whose file wasn't kept (`not masked: output not kept`), or isn't text (`not masked: not text`), is compared unmasked. The report's second line names the mask file, its sha256 and how many stages and values it changed; the JSON carries `mask` (file, sha256, rules), `masked` on each row it read, and `settings_masked`. A comparison equal only once masked reads `identical after masking`, exit 0.

**A key one run has and the other doesn't** (a config key a release added with its default, say) is a change, and a mask removes it only when a `fields` pointer names it exactly. The row then reads `same after masking (…), masking a key only one run has: /policy/redaction (only in candidate)`, and the mask line under the verdict lists every such key: masking one says its value doesn't matter, so the report keeps it in view.

**`onetrace mask suggest A B`** prints a candidate mask from two runs' differences: each setting with `version` as a word of its name (`instrument.version`, `assertions.constants.app_version`; never `conversion_rate`) that differs, as one rule when it differs in every stage your code decorated, and each value kind that differs inside a differing kept text output, with where it was seen, and what `diff` would read with it. It writes no mask file: review each line (a version change can be the real cause), then save it. `--json` prints the mask alone, ready to save. Of fields, only one named like a version that changed is suggested (`/app/version`); which other fields are noise is your call. To help with that call, it lists the fields that differ in each JSON output where the comparison opens a difference, by pointer and how (`changed`, `only in baseline`, `only in candidate`), never by value, under `fields_to_review`. `onetrace doctor` adds a note (never a fix; its exit is unchanged) when the most recent run's outputs hold timestamps, uuids, run ids or absolute paths, naming the stages and the mask line that leaves them out; a fixed date or path in an output doesn't change between runs, so the note says where those values change, not that they always do.

**`onetrace doctor --entrypoint FILE`** (0.3.1) checks that the most recent run under `runs/` was recorded through `FILE`, the file holding the `@ot.run` function production calls. Every receipt of a decorated run records the sha256 of its manifest as its instrument's `manifest_digest`: that file, unless the `@ot.run` passes `manifest=`, in which case pass the file `manifest=` names. A run recorded through a separate runner or test harness records another file's. A mismatch is a fix: another file recorded the run, or `FILE` changed after it, and the fix line says to record a run the way production runs it after your last edit. Without `--entrypoint`, doctor checks nothing of this.

A mask never changes a record, a verifier result, a refusal or a could-not-check. The runs' arguments (the inputs-differ line) are compared by digest and are not masked. `--text` shows the outputs' text as recorded, unmasked.

#### The text that changed: `diff --text`

```bash
onetrace diff runs/monday runs/tuesday --text [--stage NAME]... [--all] [--context N] [--max-lines N] [--no-color]
```

Both runs are verified first. A run that doesn't verify is refused, and no text from either run is shown. A node *differs* when anything compared differs between the runs: its outputs, its inputs, or any setting, the corpus link included. Which nodes are shown:

- **By default:** every `FIRST DIFFERENCE` node, then every node downstream of it, in order. A downstream node whose outputs are the same is shown as one line, `<node>: outputs unchanged (reconverged)`, with its settings line if a setting changed, its corpus link line if the link changed, and no text.
- **`--all`:** also the nodes that differ but aren't downstream of a first difference, for example a stage whose settings changed while its output didn't. Each gets its settings line and no text.
- **`--stage NAME`:** only the named nodes. A name that matches no node is reported.

**Chunk text is opt-in.** A splitting stage that returns `ot.chunks(pairs)` records a chunk index: each chunk's id, document, length and digest, and no text. With `ot.chunks(pairs, keep_text=True)` (or the LangChain adapter's `--keep-text`), each chunk also keeps its text, and `--text` shows a changed chunk's text diff. The text is shown only after checking that the digest of its bytes equals the chunk's digest; otherwise it says `text does not match its digest; not shown`. Run folders store each stage's outputs, so treat them like logs that may contain sensitive data. `keep_text` adds the chunk text to them.

**Input text is opt-in too** (section 16.19). A stage with `keep_inputs` stores that input's text in plain form in `inputs/`. Keep inputs only when you would also keep them in a log. `trust="secret"` inputs are never kept. **Removing the text is allowed:** delete `inputs/`, or one file in it, and the run still verifies, because a receipt holds each input's digest, never its text; `--text` then reads `not kept: file missing` for that input.

`--text` never changes a verdict or the exit code. `--json` prints the same comparison as one JSON document, `onetrace-textdiff/0.3`, whose fields and escaping are defined in [the textdiff format](textdiff-format.md).

#### Following a changed corpus link

When the first difference is at a node whose corpus link changed, `--text` follows the link one hop into the two linked ingest runs, under its own heading, `traced into the linked ingest runs: <run id> -> <run id>`:

- **They are found by chain head,** never by name or path, one level down in `--runs-root DIR` (default: the folder holding the baseline).
- **Both are verified first.** One that doesn't verify is named, and nothing from it is shown.
- **They are compared with the same rules,** starting with a sentence such as `started at: load — 1 of 214 documents changed (help-centre/warranty.md); settings unchanged`.
- **One hop only.** If the ingest runs' own corpus link changed, that is named and not followed.
- **When the trace can't be made,** it says why (not found, did not verify, a side with no link) as `COULD NOT CHECK` for the trace alone.

`--no-follow` turns the trace off.

The linked ingest runs' verifier calls are written beside the report, as the two runs' are, under `corpus/baseline/<path>/` and `corpus/candidate/<path>/`, where `<path>` is the run's folder relative to `--runs-root`. For an ingest run that doesn't verify, each call gives only its command, its exit code and the reason line, and none of the verifier's output. A run that isn't found has no calls.

Without `--text`, `diff --follow-corpus` makes the same trace and prints the ingest runs' own ladder of stages and digests after the report. It never shows stored ingest text: that needs `--text`, and the two flags aren't given together. The JSON report carries it under `corpus_trace`:
- `notes`: the plain lines, such as `trace: no corpus link recorded at <node>; …`;
- `traces`: the entries described in [the textdiff format](textdiff-format.md#a-corpus-trace). A traced entry carries `report`, the ingest pair's own `diff` or `localize` report with paths relative to `--runs-root`, in place of `nodes`.

The trace never changes a verdict or the exit code.

### 10.2 `localize`: where it went wrong

```bash
onetrace localize runs/tuesday                        # one run: first unclean stage
onetrace localize runs/monday runs/tuesday --out r    # two runs: first difference and its cause
```

On one run, an unclean stage is a refusal, an error, or a failed check.

On two runs, the cause names what changed at the first difference:
- the instrument;
- its config and constants;
- the **corpus link** a query run records with `ot.corpus_from`, by digest: `corpus link changed: <digest> -> <digest>`.

`diff` annotates a changed corpus link on every node that carries one, the way it annotates a changed setting.

When nothing recorded differs at the first difference (the same instrument, version and settings), the cause says so in one sentence (0.3.0): `no recorded cause`, whether the step's instrument declares itself not rederivable (a model call, say, whose output can vary on its own), and the next later step with a recorded change, or `nothing recorded changes after it`. On a run recorded with a topology, the next change is one downstream of the first difference (reachable over the observed edges), never a change on another branch.

#### One output's path: `localize --for`

```bash
onetrace localize runs/monday runs/tuesday --for check --out r
```

A release changes many things at once, and the first difference in chain order is often one the output you care about never reads: a version string, a new config key, a reworded document that chunks to the same words. `--for STAGE` (0.3.1) starts from that stage's output instead and asks: where does the change it shows start?

- **The path** is the stage and every stage it reads from, followed back over the links both runs recorded. On a run recorded with a topology, those are its edges. On a run read as linear, they are its receipts' own: a stage reads from each earlier stage whose recorded output it records as an input (by digest), and from a stage a `receipt:` input names. A link either run recorded is followed.
- **A linear stage that can't be traced** by its inputs (it records none, or one no earlier stage produced that isn't marked `externally-sourced` or `operator-authored`) takes every stage before it onto the path, and the report says so: `'check' records no input it can be traced by, so every stage before it is on the path`. A stage that reads something it never recorded can't be followed; the record doesn't show it.
- **The walk** goes back from the stage only through stages whose output differs, over links that carry a difference. A differing stage that no such link leads into is an **origin**: the change starts there (one per branch that changed). A difference on the path that the walk never reaches stopped before the stage (its effect reconverged, as a reworded document does once chunked), and is listed as not reaching it.
- **Which links carry a difference.** A run recorded with `@ot.run(link_parts=True)` records, for each stage, which of its arguments came from which earlier stage, whole or as a part (the `onetrace.links` constant). A link that passed only parts whose recorded digests are the same in both runs carries nothing, and is not followed, unless the stage was also given an input that differs and that no recorded link names (a string or number taken out of a return): that input may have come from the earlier stage, so the link is followed and the stage stays an origin as well, naming the input. Otherwise: a stage that reads `config["policy"]` is an origin of its own change even when another part of the config changed. Without those records, every link from a differing stage is followed, so where two changes meet, a stage that changed and also reads a differing stage, the walk goes on past it; name the noise in a mask to see the second change.
- **An input that differs at an origin.** An origin that was given an input that differs, which no recorded link traces to a stage, says so: the change may come from there. An input the record links, as a part, to a stage the comparison reads as the same once masked (a config section that gained a masked key, say), at the same place in both runs, is not counted. Nothing is matched by value: an input no recorded link names is always counted.
- **The answer** prints first: where the change starts, what doesn't reach the stage, the path with each stage's reading (`same`, `differs`, `COULD NOT CHECK`), and under each origin the recorded change there, or the no-recorded-cause sentence. The full ladder follows, as without `--for`; its first differences are the run's, so a stage below a reconverged difference still reads `downstream` there.
- **The stage's own output must differ.** One that doesn't reads `its output does not differ between the runs; nothing to trace`, exit 0. A stage the walk meets that couldn't be checked makes the answer `COULD NOT CHECK`, exit 4, naming it. A name neither run has, or a bare name several instances share, is a usage error, exit 2, naming the choices.

The verdicts are the ladder's own; `--for` only reads them along one output's path, and changes none. The JSON report carries the answer under `for` (`origins`, `not_reaching`, `given_differs`, `path`). The walk is as good as the links the runs recorded: a stage given a value built between stages (a new dict made from two returns), or a string or number taken out of a return, has no link for it, and the walk can stop early there, naming the stage as an origin given an input that differs.

#### Runs on the same input: `onetrace match`

```bash
onetrace match runs/tuesday [--runs-root runs] --out r
```

`match` (0.3.1) lists the runs given the same recorded inputs as this one, so a comparison can be between runs on the same job. It looks one level down in `--runs-root` (default: the folder holding the run), compares each run's arguments (its intake stage's inputs, as above) with this run's by digest, and checks each run that matches with the verifier. Only a run that verifies is listed, newest first by its first receipt's recorded start, each with the `onetrace diff` command that compares it with this one (both paths resolved, as the runs are found, so a linked folder or a Windows short folder name is not mixed with its other form); a run that matches but doesn't verify, or couldn't be checked, is named under `not listed`, with why. Exit 0 when one or more runs are listed, 1 when none, 3 when this run doesn't verify, 4 when it has no intake stage to match on or couldn't be checked. Nothing is read but the receipts.

Each verb prints and writes a carried verifier line that repeats across receipts once, with a count (`[NOT-RUN] originality on 3 receipts -- ...`). `--verbose` prints and writes each line; the JSON report always keeps each.

`localize A B --follow-corpus` follows a changed corpus link one hop, as `diff` does, and reports where the two linked ingest runs first differ and why, after its own report. It never shows stored ingest text (see [following a changed corpus link](#following-a-changed-corpus-link)).

The corpus link is compared on run records only. A query run's retrieval stage is a run record, so the per-document receipts of the two-root form are not compared for a corpus link.

#### A straight-line run against a graph run

A run written with the Recorder API records its stages in a straight line, with no edges; the same
stages decorated record the graph they ran in. When the two declare the same stages, `diff` and
`localize` compare them node by node. A straight-line run's receipts may still record what a stage
read (`ctx.read_receipt`): when they record any read, those reads are its edges, so a stage that
read something else than in the graph run is a difference and `--for` follows the read it recorded;
the line under the verdict says `shape: the baseline records no edges in its manifest (a
straight-line run); its stages are compared under the reads its own receipts record`. Only when it
records no read at all are its stages compared under the edges the graph run recorded, and the line
says `... compared node by node under the edges the other run recorded`. Two runs that declare
different stages are not comparable, as before.

#### How long each stage took: `--durations`

`diff A B --durations` (and `localize A B --durations`) adds a time section: each node's duration
in both runs, read from its receipts' own `time.started` and `time.ended`, and the note "slower"
where the candidate took at least `--slow-factor` times as long (default 2) and at least
`--slow-min-ms` milliseconds more (default 1000). Both conditions, so a stage going from 2 ms to
9 ms isn't called slower. The JSON report carries it under `durations`, every number a decimal
string.

**Timing is a view, never a verdict.** The verdict, the exit code and every other part of the
report are byte for byte what they are without the flag. A run's times depend on the machine, its
load and the network, so a slower note says what the receipts recorded and nothing about why.
A receipt whose times can't be read shows `-`.

#### Many runs side by side: `onetrace drift`

`onetrace drift RUN1 RUN2 RUN3 ... --out DIR` compares each run with the first, exactly as `diff`
compares two (the same verifying, the same mask, the same verdicts), and lays the comparisons side
by side: one line per run with its verdict against the run it was compared with, then one row per
node with the verdict of each comparison. `--against previous` compares each run with the one
before it instead, which shows where a change came in and whether a later run went back.

The exit code is that of the first comparison, in run order, whose exit code isn't 0 (1 diverged,
2 not comparable, 3 refused, 4 could not check), or 0 when every run reads identical. A run the
verifier refuses is named on its line and its comparison reads refused; the other comparisons are
still made. With `--durations`, the time section has one column per run, and every node slower
than in the run it was compared with is listed.

### 10.3 `reproduce`: re-run and compare

`reproduce` re-executes recorded stages and compares their outputs with the record. It needs a **runner file** that says how to run your code:

```json
{"format": "onetrace-runner/0.1",
 "code": "pipeline.py",
 "argv": ["{python}", "-B", "{code}", "{out}"],
 "cwd": "{code_dir}",
 "timeout_seconds": "600"}
```

```bash
onetrace reproduce runs/monday --runner runner.json --out reports/repro          # whole run
```

- `{python}` is the interpreter that is running `onetrace`. Run `reproduce` from the virtual environment that has your pipeline's dependencies installed.
- `{out}` is a fresh folder that `reproduce` chooses. Any further arguments your code needs, such as a run id, go in `argv` as plain strings.
- **Re-running a single stage is not available in 0.1.x.** `onetrace reproduce <run> <stage>` is accepted, but it always reports that stage as `COULD NOT CHECK` (exit 4), naming the reason. Reproduce the whole run instead.

- The code's sha256 must equal the recorded `manifest_digest`, and the instrument identity must match exactly. Otherwise the result is `COULD NOT CHECK`, never "close enough".
- Each stage is reported as `REPRODUCED`, `DIVERGED` or `COULD NOT CHECK`.
- Stages marked `rederivable="false"` are expected not to reproduce.

### 10.4 A summary for a pull request: `onetrace summary`

```bash
onetrace-verify --json runs/candidate > verify.json
onetrace diff runs/baseline runs/candidate --out r
onetrace summary r --verify-json verify.json --title "release check"
```

`onetrace summary DIR` (0.3.1) prints a short Markdown summary of what a `diff` or `localize` run wrote in `DIR`, and of `onetrace-verify --json`'s output if you pass it: the verifier's result and any failing row, the comparison's verdict and first difference, and, when the report holds them, the inputs-differ line, the mask used, the `--for` answer, the outputs that changed at the end and the cause sentence. It runs nothing and decides nothing: it repeats the verdicts already reached, and exits 0. Every string taken from a record is shown as inline code, so a record can't add a link, an image, HTML, a mention (`@name`) or an automatic link (a bare URL or address) to a comment; control characters (C0 and C1) are shown as `\xNN`, and format characters (right-to-left overrides, zero-width marks) as `\uNNNN`, so no text is reordered or hidden. The title you pass is escaped for Markdown, with `@` written as an entity. Its first line is the marker `<!-- onetrace summary -->`, so a CI job can find and update its own comment.

### 10.5 Exit codes and reports

| Command | 0 | 1 | 2 | 3 | 4 |
|---|---|---|---|---|---|
| `diff`, two-run `localize` | identical (or identical after masking) | diverged | not comparable, or a mask file refused | refused | could not check |
| `localize A B --for STAGE` | the stage's output does not differ | located on its path | not comparable, or `--for` names no stage | refused | could not check |
| `match` | runs listed | none listed | n/a | refused | could not check |
| one-run `localize` | clean | located | n/a | refused | could not check |
| `reproduce` | all REPRODUCED | any DIVERGED | n/a | refused | any COULD NOT CHECK (none diverged) |
| `onetrace-verify` | `PASS` | `FAIL` | `NOT VERIFIED`, or refused (below) | n/a | n/a |

For `onetrace-verify`, exit 2 is never a pass and never a fail. It has three causes:
- a receipt whose own format this verifier does not implement, with no check failing anywhere. The summary line reads `NOT VERIFIED (receipt format not implemented: N receipt(s))`.
- a manifest whose own chain format it does not implement, or that it cannot read. This is reported as a `[REFUSED]` line before any row.
- a usage error, such as a missing path.

A `FAIL` anywhere always outranks `NOT VERIFIED`, giving exit 1. With `--json`, the exit code is the same, and `summary.result` reads `NOT VERIFIED` or `REFUSED` for the first two causes.

`diff`, `localize` and `reproduce` read every run as `onetrace-verify` does, before anything else (0.3.0):
- a run it shows wrong (its exit 1) is **refused**, exit 3, naming the failing rows;
- a run it could not check (`NOT VERIFIED`, its exit 2) is **could not check**, exit 4, naming the run and the verifier's reason. No verdict is read from a run whose receipts were not all checked, so such a run never reads `identical`. The one format the verbs read on purpose without the verifier implementing it is the preview format (`stage-receipt/0.0-preview`, Appendix C); its runs compare as before, with the line `COULD NOT CHECK — draft-00 checks not performed` carried.

A refusal always wins: a pair with one refused run and one that could not be checked exits 3.

Every command writes a JSON report and a text report into `--out` (default `onetrace-report/`), plus the verifier's rows for each input run. Use `--quiet` to skip printing.

---

### 10.5 Seeing what changed

The repository's warranty example is a help-centre corpus in two versions, where one sentence in `warranty.md` says "twelve months" in v1 and "six months" in v2. Each version is ingested (`load`, `split` with `ot.chunks(..., keep_text=True)`, `index`), and a query pipeline (`retrieve`, `prompt`, `answer`) runs over each, linked to its own ingest run with `ot.corpus_from`. The four runs are in `tests/fixtures/warranty/`. From that folder:

```bash
onetrace diff runs/query-v1 runs/query-v2 --text --no-color
```

prints this before the ladder report (this is the test suite's committed transcript, byte for byte):

```text
first difference: retrieve — output text changed; settings unchanged; corpus link changed; input help_centre changed

== retrieve (FIRST DIFFERENCE)
settings: none changed
corpus link changed: 67ffdb182dfb (ingest-v1) -> 9f88f639a4a5 (ingest-v2)
input help_centre (operator-authored) changed: cec075f03ffe -> 1e4f8fbeaa98; content not stored
--- return.json
hits: 0 entered, 0 left, 0 moved; 1 changed; 1 unchanged
~ hit help-centre/warranty.md#0001 #1 changed: b500694d27e1 -> 5ceb7e1a7fba
  - The warranty covers twelve months from the date of purchase.
  + The warranty covers six months from the date of purchase.
  ~ The warranty covers [-twelve-]{+six+} months from the date of purchase.
    It covers manufacturing faults, not accidental damage.

== prompt (downstream)
settings: none changed
--- return.json
- "Answer from these passages only.\n- The warranty covers twelve months from the date of purchase.\nIt covers manufacturing faults, not accidental damage.\n- You can return an unused item within 30 days of delivery.\nRefunds go back to the original payment method within 5 working days.\nQuestion: How long does the warranty cover?\n"
+ "Answer from these passages only.\n- The warranty covers six months from the date of purchase.\nIt covers manufacturing faults, not accidental damage.\n- You can return an unused item within 30 days of delivery.\nRefunds go back to the original payment method within 5 working days.\nQuestion: How long does the warranty cover?\n"
~ "Answer from these passages only.\n- The warranty covers [-twelve-]{+six+} months from the date of purchase.\nIt covers manufacturing faults, not accidental damage.\n- You can return an unused item within 30 days of delivery.\nRefunds go back to the original payment method within 5 working days.\nQuestion: How long does the warranty cover?\n"

== answer (downstream)
settings: none changed
--- return.json
- "The warranty covers twelve months.\n"
+ "The warranty covers six months.\n"
~ "The warranty covers [-twelve-]{+six+} months.\n"

traced into the linked ingest runs: ingest-v1 -> ingest-v2
started at: load — 1 of 3 documents changed (help-centre/warranty.md); settings unchanged

== load (FIRST DIFFERENCE)
settings: none changed
input help_centre (operator-authored) changed: 5ae58f3b8452 -> bf96196b62ce; content not stored
--- return.json
    },
    {
      "id": "help-centre/warranty.md",
-     "text": "# Warranty\n\nThe warranty covers twelve months from the date of purchase.\nIt covers manufacturing faults, not accidental damage.\n"
+     "text": "# Warranty\n\nThe warranty covers six months from the date of purchase.\nIt covers manufacturing faults, not accidental damage.\n"
~     "text": "# Warranty\n\nThe warranty covers [-twelve-]{+six+} months from the date of purchase.\nIt covers manufacturing faults, not accidental damage.\n"
    }
  ]

== split (downstream)
settings: none changed
--- return.json
chunks: 1 changed, 0 added, 0 removed; 5 unchanged
~ chunk help-centre/warranty.md#0001 changed:
  - The warranty covers twelve months from the date of purchase.
  + The warranty covers six months from the date of purchase.
  ~ The warranty covers [-twelve-]{+six+} months from the date of purchase.
    It covers manufacturing faults, not accidental damage.

== index (downstream)
settings: none changed
--- return.json
        "not",
        "of",
        "purchase",
+       "six",
        "the",
-       "twelve",
        "warranty"
      ]
    }
```

Reading it from the top:

- **The first line is a sentence:** where the runs first differ, whether its output text changed, whether any setting did, and which inputs did.
- **`retrieve` is the first difference.** No setting changed, but its corpus link did, on a line of its own: the query runs read different ingest runs, named by run id once the trace reached them. Its input `help_centre` changed too. An input's name is its parameter's name, so it is spelled as Python spells it; `help-centre/warranty.md` is a document's path, which keeps its own spelling. Inputs are fingerprinted, so only their digests are shown.
- **The changed words are marked** `[-removed-]{+added+}` on a `~` line under each changed line pair, here `twelve` to `six`. A chunk index and a hits list are compared entry by entry, by id.
- **`prompt` and `answer` differ too:** they are downstream of `retrieve`, and their settings did not change.
- **The trace follows the changed corpus link one hop** into the two ingest runs, verified first, and compares them the same way. It starts at `load`, where one of three documents changed, and shows the sentence changing in `warranty.md` and then in its chunk.

**Which settings a comparison names.** A setting is named, with both values, only when it was recorded with `ot.constant()` (16.14): `constants.temperature: 0.2 -> 0.7`. A stage's instrument config is recorded as one digest, so when it changes the comparison says `instrument.config_digest` changed, not which key. To see a setting by name in `diff --text`, in its JSON and on the console's Compare page, record it with `ot.constant()`. A changed tool (`instrument.id`, `kind` or `version`) is counted on the `settings:` line too.

The same comparison is available as one JSON document with `--json`. Its fields are described in [the textdiff format](textdiff-format.md), and its JSON Schema is `docs/textdiff-format.schema.json`.

What this does not claim:
- *The text diff shows what recorded outputs contained. It does not say why they changed, or that either version was correct.*
- *Inputs are fingerprinted, so their content is not shown.*

## 11. Using onetrace in CI

```yaml
# example: fail the build if the pipeline's record is broken or its output drifted
- run: python pipeline.py runs/ci run-ci-${{ github.run_id }}
- run: onetrace-verify --require-artifacts runs/ci
- run: onetrace diff baselines/golden runs/ci --out reports/diff   # exit 1 means the output changed
```

Keep a known-good run as `baselines/golden`. When a change is intended, regenerate the baseline in the same pull request, so the diff is reviewed rather than hidden. `onetrace-verify --require-artifacts` checks your output files too, and fails if they can't be checked (section 8.3), so no separate script is needed.

---

## 12. Common errors and what they mean

| Message (abridged) | Cause | Fix |
|---|---|---|
| `emission policy must be declared as one of ('fail-closed', 'fail-open')` | missing or misspelled `policy` | pass one of the two |
| `... already holds a run; a record is never overwritten` | the output folder holds a finished run (`close()` releases `.lock`, so reopening it reads as this, not as a lock conflict) | use a new folder per run |
| `another process already holds this run folder` | a second process genuinely still emitting into that folder, or a stale `.lock` left by a process that crashed before calling `close()` | one process per folder, and a fresh folder per run |
| `stage 'X' is not the next declared stage (expected Y)` | stages ran out of declared order | call them in the declared order, or fix `declared_stages` |
| `version 'latest' does not pin a build` | an unpinned instrument version | use the exact library version or model snapshot |
| `... nan is not a number that can be recorded (NaN and Infinity have no RFC 8785 form)` | NaN or Infinity in a config, constant or assertion | record what it stands for as a string (`"unbounded"`), or leave the setting out |
| `JSON number at $.… : use a decimal string` (`write_json` only) | a number in a `write_json` payload | convert it to a string: `str(x)` |
| `write_json needs a JSON object at the top level` | a list or a string passed to `write_json` | wrap it in an object |
| `assertions [...] declare no constant` | assertions with no `ctx.constant` in the stage | declare the constants the stage ran under |
| `trust class 'x' is not one of (...)` | a wrong trust class | use `operator-authored`, `model-generated` or `externally-sourced` |
| input refused as `secret` | `trust_class="secret"` | keep secrets out of the record entirely |
| `StoreExhausted` | `max_run_bytes` reached | raise the bound, or write less |
| `max_run_bytes=... is below ..., the smallest a real receipt for this run's own declared stages could ever be` | the bound was set below what even one error receipt needs, measured from your own run at construction | raise the bound to at least the number named |

---

## 13. API summary

```python
import onetrace
onetrace.__version__                         # "0.6.1"
onetrace.digest(path_or_bytes) -> "sha256:<hex>"
onetrace.current_stage()                     # the running step's context (its .name, its .write_json), or None
onetrace.model_from(run, stages=[...], serve_stage=None)   # a model link, in the corpus-link form (16.20)

from onetrace.emit import (Recorder, Instrument, Refusal,
                           EmissionRefused, StoreExhausted, current_run, current_stage)
from onetrace.verify_pin import VerifyPin

Recorder(out_dir, declared_stages, manifest, policy, anchor_reason=None, run_id=None,
         topology=False, declared_edges=None, boundaries=None, instruments=None,
         verify_pin=None, max_run_bytes=None)
  .stage(name, instrument=None, instance=None)        # decorator, or `async with`
  .run_stage(name, instrument, fn, *args, **kw)
  .run_node(name, instance, instrument, fn, *args, **kw)
  .boundary(name, kind, note=None)
  .gap(stage, reason, kind="unwritten receipt")
  .close()
  .run_id                                             # the id in use

Instrument(id, kind, version, config, rederivable="true", rederivable_note=None)

ctx (StageContext):
  .read_external(path, media_type, name=None, trust_class="externally-sourced") -> bytes
  .read_memory(data, media_type, name, trust_class="externally-sourced") -> bytes
  .reference(path, media_type, name=None, trust_class="externally-sourced", digest=None) -> Artifact
  .read(artifact) -> bytes          .read_json(artifact) -> object
  .read_receipt(node) -> dict
  .write(name, data, media_type, trust_class="operator-authored") -> Artifact
  .write_json(name, obj, trust_class="operator-authored") -> Artifact
  .constant(key, value)             .assertion(key, value)   # value: str, int, float or bool (rule M)
  .corpus_manifest(digest, member=None, preimage=None, proof=None)

Refusal(reason, detail=None)        # raise inside a stage to decline

VerifyPin(verifier=None, expect_sums=None, per_receipt=False)
```

Commands: `onetrace-verify [--require-artifacts] [--json] [--headers FILE] [--tsa-cert FILE --tsa-root FILE] [--trust FILE] [--require-signature] [--require-trusted] [--require-complete] <run>`, `onetrace anchor <run> --source rfc3161|opentimestamps` (section 15), `onetrace diff A B`, `onetrace localize A [B]`, `onetrace reproduce <run> [stage] --runner R`. The common options are `--out`, `--quiet`, `--stages` and `--verifier`. `onetrace-verify -h`/`--help`/`--version` print usage and the version and exit 0.

---

## 14. Known limitations

- Records are **hash-chained**; signing (section 17) and anchoring (section 15) are optional. Without them, the chain alone does not show who wrote a record. To show a record existed by a given time, anchor its chain head (section 15); otherwise keep the chain head somewhere independent.
- Framework "adapters" are worked examples in the source distribution, not importable modules.
- The format is version 0 and may change before version 1. Every record carries its format version. The manifest's own chain format is checked, and an unknown one is refused by name, never guessed. **A receipt's own format was not checked in 0.1.1** (0.1.0 behaved the same): only its presence was checked, not its value. **From 0.1.2 it is checked.** A missing format, one that is not a string, or one that is not a `stage-receipt/<major>.<minor>` label is a `FAIL`. A version this verifier doesn't implement (a major other than 0, or a minor that isn't ASCII decimal digits) gets one `NOT-RUN` row naming it: that receipt's digest against the manifest and its chain link are still checked, but none of its content is (canonical form, fields, originality, output files). The summary then reads `NOT VERIFIED (receipt format not implemented: N receipt(s))` and the exit code is 2, unless a check failed anywhere, which gives `FAIL` and exit 1 (section 10.5).
- `onetrace-verify`'s artifact check only applies when it can recognise a run's own `artifacts/<stage>/<output>` layout. A run built by hand, or with a different layout, reads `NOT-RUN` for its files (section 8.3). Receipts don't record where an output file is stored, so the check depends on this SDK's folder convention; a later format version will record the path.
- `reproduce` re-runs whole runs only (section 10.3).
- Licence: Apache-2.0.

### Fixed since 0.1.0 — if you have run folders from 0.1.0

- **`onetrace-verify` did not check `artifacts/` files at all** in 0.1.0; as of 0.1.1 it does (section 8.3). This makes the verifier **stricter**, not looser: a run whose output files were edited or deleted after the fact, which passed under 0.1.0, now correctly fails.
- **A concurrent pipeline's output files could land under the wrong numbered folder** in 0.1.0: `ctx.write`'s target directory was chosen from a stage's provisional position, before the real one was decided under the lock, and the two could disagree once two stages genuinely raced. The receipts themselves were never affected — only where the output *files* ended up on disk. **If you have run folders emitted concurrently by 0.1.0**, re-verifying them under 0.1.1 may now show `[FAIL]` rows naming misplaced output files that 0.1.0's own verifier could not see. This is 0.1.1 correctly reporting a real, pre-existing mismatch, not a new defect in the run itself.
- **`onetrace-verify -h`/`--help` were previously treated as a path**, and a run with a missing or malformed receipt file made it stop with a Python traceback instead of a clean `[FAIL]` row. Both are fixed in 0.1.1: usage/version print and exit 0; any file problem the manifest names gives a `[FAIL]` row (or, for the manifest itself, `[REFUSED]`), never a traceback.
- **`reproduce` could not run your code from a virtual environment on Linux or macOS.** `{python}` resolved the venv's interpreter link to the system Python, which doesn't have your packages, so every stage read `COULD NOT CHECK`. Fixed in 0.1.1: `{python}` is the interpreter running `onetrace`, exactly as invoked.
- **A config, constant or assertion value had to be a string**, including numbers (`"3"`, not `3`) — a JSON number anywhere in a receipt was refused outright. As of 0.1.1, a Python `int` is accepted directly and recorded as its own decimal string (section 5.3); as of 0.2.0, a `bool` and a `float` are too (rule M, section 16.16). `write_json` payloads are unaffected: a number there is still refused, now with a clearer message naming the output.

## 15. Anchoring a run

`onetrace.anchor.add_anchor(run_dir, record)` stores each anchor as its own file, `anchors/<n>.json`, beside the run's receipts. The writer holds `anchors/` open while it links the record in and reads it back, and it reports a record stored only when it has confirmed that the record is there, in this run's folder, with the bytes it wrote. If it can't confirm that, it says so ("stored as N.json; could not confirm it landed in this run"), and you should check the folder before relying on it. It never reports a record stored that it hasn't confirmed.

### 15.1 Run folders on a network share

- **An SMB share of an NTFS volume (Windows):** anchors are stored and confirmed. This was measured through the loopback admin share (`\\localhost\C$`). Windows can't open a file by its id over SMB, so the writer opens the stored name relative to the directory it holds. Over SMB the server resolves that "relative" open by path; what keeps it safe is that the writer then checks the file it opened is the file it wrote, by its file id.
- **The limit over a share.** Over a network share, someone who can write to the run folder on the server during the write can make the record land in another run. The writer then reports that nothing is claimed, and the stray record reads FAIL there. On a local disk this can't happen. To anchor a run folder that others can write to, anchor a local copy of it.
- **A ReFS volume, local or shared:** not measured. A record there may not be confirmed: the writer refuses to confirm a file whose id is wider than 64 bits, which ReFS can use.
- **A share without hard links** (for example WSL's `\\wsl.localhost\...` share): the record is refused before anything is claimed, because the writer needs a hard link so that no reader ever sees a partial record.
- **Other SMB servers** (for example Samba) haven't been measured. If the writer can't confirm a record there, it says "not confirmed", never "stored".

### 15.2 Someone else writing in the run folder at the same time

The writer links the file it wrote itself, not whatever is at the temporary file's name. On Windows the link is made from the file's own handle, and while the writer holds it, the temporary file can't be renamed away. On Linux the link is made through `/proc/self/fd`.

On Windows, someone can make the stored record a reparse point after it is written, which keeps the file's id. The verifier then reads the record as FAIL. If the reparse point is set before the writer reads the record back, the writer says "stored as N.json; could not confirm…", naming the reparse point: it doesn't report that record stored. If it is set after the read-back, the writer reports the record stored, and only the verifier's FAIL shows it.

On a system without `/proc/self/fd` (macOS, for example), the temporary file is linked by its name. Someone with write access to the run folder during the write can then put a copy in its place. The writer detects this and refuses with "nothing is claimed", but the copy stays in `anchors/`. This behaviour was measured on Linux with the `/proc` route turned off; macOS itself hasn't been measured. **Anchor a run folder that nobody else is writing to.**

### 15.3 What the verifier reports for an anchor

Each file in `anchors/` gets one row, beside the receipts' own rows. It reads PASS, FAIL or NOT-RUN, with its reason:

- **PASS:** the proof is valid, it commits to this run's chain head, and it comes from the source you pinned.
- **FAIL:** something is shown to be wrong. The verifier's result is then FAIL.
- **NOT-RUN:** a check that can't be computed here. It is never a pass, and it leaves the verifier's result as it would be without the anchor. The reasons are:
  - "needs block headers": an OpenTimestamps proof, with no header file for its block;
  - "pending: not yet in a block": an OpenTimestamps proof the calendar hasn't put in a block yet;
  - an operation or attestation this verifier can't compute, named;
  - a `method` this verifier doesn't implement, named;
  - without `onetrace-verify[crypto]`, the install line.

**The order of the checks is fixed.** The record's structure, its size and nesting caps, and its binding to this run's chain head always come first, with or without the extra and whatever the method. A failure there is FAIL. So a malformed record, or one for another run, is never shown as NOT-RUN.

**The record's format is `stage-receipt-anchor/0.1`.** A record written as `onetrace-anchor/0.1`, the name before 0.2.0, is FAIL, and the row's `fix:` names the new one.

**`asserted_time`** is the time the proof attests:
- it is `null` while the proof attests no time yet (a pending OpenTimestamps proof), and only then;
- `null` beside a proof that attests a time is FAIL, and so is a time beside a proof that attests none;
- any RFC 3339 time with `Z` or a UTC offset is accepted, and times are compared as instants: `2024-09-22T12:13:20+02:00` and `2024-09-22T10:13:20Z` are the same time.

**onetrace's own record shape.** The approved −01 text leaves these open; this is what onetrace writes and accepts:
- **`source`** is `{"tsa_certificate_sha256": "<64 lowercase hex>"}` for `rfc3161`, and `{"calendars": ["https://…", …]}` (a non-empty list) for `opentimestamps`. For a method it doesn't implement, the verifier doesn't read `source`.
- **Strictness:**
  - a member the record format doesn't define is FAIL;
  - a file in `anchors/` that isn't named `<n>.json` (n from 1, no leading zero, lower case) is FAIL;
  - an `anchors/` that is a link (a symlink or a junction) is FAIL, and nothing in it is read.

### 15.4 Anchoring with RFC 3161 (a time-stamp authority)

An RFC 3161 time-stamp service (TSA) is the method to use where no outside calendar can be reached: its answer is checked against the service's certificate with no network call. With no outside access, point `--tsa-url` at a time-stamp service you run yourself and pin its certificate; onetrace has none built in. It needs the `[crypto]` extra (15.6).

```
onetrace anchor runs/<run_id> --source rfc3161 --tsa-url https://tsa.example/tsr \
    --tsa-cert tsa.pem --tsa-root root.pem
```

- The request is a time-stamp query over the chain head: SHA-256 over its 32 raw bytes, with a nonce. This is the only onetrace command that contacts a network, and only when you run it.
- Before anything is stored, the TSA's answer is checked the way the verifier will check it, against the certificate and root you pinned. An answer that isn't granted, carries another nonce, is for another chain head, or doesn't verify is refused, with nothing stored and a `fix:` line.
- The record keeps the TSA's answer byte for byte, names the pinned certificate by its sha256, and carries the token's own time as `asserted_time`.
- Exit status: 0 stored, 1 refused (nothing stored), 2 a usage error (nothing asked of any source), 3 stored but not confirmed (15.1, 15.2). For `--upgrade`, 0 also when every upgrade is already stored and nothing is still pending (it names the record that holds it, and stores nothing).

To check the anchor, pin the same certificate and root:

```
onetrace-verify runs/<run_id> --tsa-cert tsa.pem --tsa-root root.pem
```

Without them, the row reads NOT-RUN: "needs the TSA certificate and root you trust". Some TSAs built on OpenSSL send their certificates in an order the parser refuses. The verifier puts only that set in order (it is not covered by the TSA's signature), and the row says so: "certificate set re-ordered, unsigned; signature unaffected".

### 15.5 Anchoring with OpenTimestamps (public calendars)

OpenTimestamps is public and free. A proof is first pending, then, after some hours, completed in a Bitcoin block. It needs no extra to write.

```
onetrace anchor runs/<run_id> --source opentimestamps \
    --calendar https://a.pool.opentimestamps.org --calendar https://b.pool.opentimestamps.org
```

- A random nonce is added to the chain head before it is hashed and sent, so a calendar never sees the chain head itself. No calendar is built in.
- The pending record reads NOT-RUN, "pending: not yet in a block".
- Later, complete it:

  ```
  onetrace anchor runs/<run_id> --source opentimestamps --upgrade --headers headers.json
  ```

  The completed proof is stored as a new record; the pending one is kept, since a record is never rewritten. Its `asserted_time` is the block's time, from your header file. You learn which block to fetch only once a calendar has completed the proof: if that block isn't in your header file yet, `--upgrade` stores nothing and names its height. Add that block's header and run it again. Every upgrade is checked before any is stored, so a refusal stores nothing. An upgrade already stored is named and not stored again; that exits 0 only when no proof is still pending.
- A header file is a JSON object mapping each block height (decimal) to its 80-byte header as 160 lowercase hex characters. It is your trust (see `non-claims.md`).
- To check a completed anchor:

  ```
  onetrace-verify runs/<run_id> --headers headers.json
  ```

  Without a header file, the row reads NOT-RUN, "needs block headers". The block's time is Bitcoin's own and can be up to two hours off: Bitcoin's own consensus rule allows a block's timestamp that much leeway. So an `asserted_time` within two hours of it agrees.

`onetrace diff` and `onetrace localize` show each run's anchors as an annotation (`anchoring [side]: …`). An anchor never changes a stage's verdict or the exit status.

### 15.6 The `[crypto]` extra, pinned

Anchor checks, and RFC 3161 anchoring, need one extra: `pip install "onetrace-verify[crypto]"` (or `onetrace[crypto]`). Its dependencies are pinned exactly, and `requirements-crypto.txt` beside each `pyproject.toml` lists every file of each by its sha256. For a hash-checked install:

```
pip install --require-hashes -r requirements-crypto.txt
pip install --no-deps onetrace-verify
```

Without the extra, each anchor still has its structure, size limits and chain-head binding checked; a failure there is FAIL, and only then does it read NOT-RUN, "install onetrace-verify[crypto] to check anchors".

## 16. Decorators

`@ot.run` and `@ot.stage` record a pipeline from two decorators, over the same Recorder described
above. This section grows with them; what is documented here is what is built.

### 16.1 What a decorated stage stores

Each call of a decorated stage is one node. Its arguments are recorded as inputs, named by
parameter. Its return value is stored as the node's output artifact, so a later comparison can
show what changed in it, not only that it changed:

- **`bytes`** (or `bytearray`) are stored exactly as returned, as `return.bin`, media type
  `application/octet-stream`.
- **Any other value** is stored as `return.json`, media type `application/json`, in the decorated
  values' encoding: RFC 8785 JSON, with numbers allowed (floats in their shortest round-trip form,
  integers up to 2^53 - 1), keys in a fixed order and no whitespace. This is not `ctx.write_json`'s
  form, which refuses numbers: the two encodings are kept apart on purpose.
- **A value neither covers** (an object with no registered encoder) stops the stage with
  `UnencodableValue`, naming the stage, the parameter and the type. It is never stored as
  `repr()`.

An argument that is an earlier stage's return value, passed on unchanged, is recorded as an edge
from that stage (`receipt:<stage>`), not as a second copy of the value.

**Stages link only when the same object is passed on whole; with `link_parts=True`, also a dict or
list inside it** (0.3.1). The link is made by identity: the argument must be the very object the
earlier stage returned. With `@ot.run(link_parts=True)`, a dict or list that is itself part of that
return (`config["policy"]`, `ctx["facts"]`, `rows[0]`, to three levels down) links too; it is
recorded twice, the edge from that stage and its own value under the parameter's name, and each
stage records which of its arguments link where, whole or at which JSON pointer (the
`onetrace.links` constant), so a comparison can tell which links carry a difference. A link says the
stage was given the same object an earlier stage returned, not where that object was made: a
module-level dict that one stage returns inside its result and another is given directly reads as
coming from the first. Anything made from a return is a new object, recorded as an ordinary input
with no edge: a slice, a copy or `list(x)`, a dict built from two returns, an equal value built
elsewhere, and any string or number, even one taken out of a returned dict (Python shares those
between unrelated places, so they are never linked). To keep a link for one of those, pass the
containing dict on and take the value inside the next stage. Part links are off by default: runs
recorded with and without them differ in their links, so turn it on for a baseline and its
candidates together (a comparison of one of each says so once, `links: the baseline records part
links …`), and approve an approved plan (`declared_edges`) again from a run that records
them.

**Trust through containers** (0.4.0, with `link_parts=True`). A part of an earlier return is
recorded with the trust class of the return it came from (that stage's `output_trust`), not the
receiving stage's `trust=`, so a `model-generated` answer passed on as a part stays
`model-generated` in the record. An argument your code built, a state dict or a prompt bundle, keeps
the stage's own class, and the constant `onetrace.held` names each earlier return, or part of one,
it holds: `[parameter, pointer in the argument, stage, pointer in that stage's return, class,
digest]`, found by identity to three levels and at most 5,000 dicts and lists, with an edge to each
stage named. Where the search stops short (more containers than that, or dicts and lists nested
deeper), `onetrace.held_cut` reads `"true"` and `diff` and `localize` say so in a `note:` line under
the verdict: a value held beyond it is not named. A held value whose digest
changed between two runs reaches its producer in `localize --for`. `onetrace doctor` notes a stage
declared `operator-authored` whose input holds `model-generated` values; its exit is unchanged.
As for part links, only dicts and lists are found, never an equal value made elsewhere, and a
string taken out of a return and put in a state is a new value: keep model text inside a dict
(`{"text": …}`) to keep it traceable.

**Lineage through a framework's state** (0.4.0). A LangGraph node is given the graph's state, a
dict the framework builds afresh for each node, so nothing it is given is the object an earlier node
returned and no edge is recorded. Decorate its nodes with `from onetrace.langgraph import stage`
instead of `ot.stage` (every option is the same): each node then records an edge from the node that
last wrote each key of the state it is given, and the constant `onetrace.state` lists each key with
its writer, the writer's output class and the key's digest. A node writes a key when its returned
mapping holds that key with a value different from the one it was given, so a node that returns the
whole state writes only what it changed. A key the framework combines (a reducer, such as
`Annotated[list, operator.add]` appending messages) keeps a row for every node that wrote it since it
was last replaced, since each one's update is in the value the next node reads; a key replaced
outright names its last writer alone. onetrace tells the two apart by what the next node is given:
exactly the writer's update, or something else. `localize --for` then follows a writer only when a
key it wrote reads differently at the node. The edges are data edges (which node's write a node read), not
the graph's control edges; an approved plan (`declared_edges`) for such a run lists data edges.
LangGraph itself is not imported by onetrace. A state that holds LangChain messages records each message's per-call `id` unless `onetrace.langchain` is imported, and a key whose digest differs on every run reads as a change from its writer every time: import it alongside.

For a value a stage reads by a way onetrace can't see at all (a shared store, a cache),
`@ot.stage(..., reads=["stage"])` records an edge from that stage's latest node in the run, and
`onetrace.reads` names it (`""` where that stage hasn't run yet, as in a loop's first pass; `diff`
and `localize` then say in a `note:` line that no edge from it is recorded). A node that changes the
state it was given in place, without returning the change, is not named as the key's writer: return
every change as an update.

**A gap, not yet fixed:** Python reuses some objects across a program: `None`, `True` and `False`,
and numbers and strings written as literals in your code. A stage that returns one of these, and a
later stage that is given the same value from anywhere else, are linked as if the second had read
the first's output. Return a value that carries the stage's data, not a bare `None`, flag or
constant.

### 16.2 Fields you didn't state

Three fields say what a value means, and the decorators never guess them:

- **`trust=`**, the trust class of the values a stage records (its arguments and its `files=`).
  A stage's own `trust=` applies; otherwise its run's `trust=`. With neither, the values are
  recorded as `externally-sourced`, the most cautious class, and the stage lists `"trust"` in
  `assertions.undeclared`. A stage whose only inputs are earlier stages' return values records no
  value under a trust class, so it lists nothing for it. The run's own arguments (the `intake`
  stage) follow the run's `trust=` the same way.
- **`rederivable=`**, whether the stage's output can be made again from its inputs. Stated on the
  stage (with an optional `note=` saying why) or on `ot.pkg`, it is recorded as stated. Stated on
  neither, the instrument records `rederivable: "false"` with `rederivable_note: "not stated by a
  person"`, and the stage lists `"rederivable"` in `assertions.undeclared`. A `note=` on a stage
  without `rederivable=` is refused when the stage is decorated.
- **`output_trust=`**, the trust class of what the stage returns: `operator-authored` for content
  your code constructed (counts, digests, an index of ids), `externally-sourced` for content that
  arrived from outside (documents, passages, a prompt that quotes them), `model-generated` for a
  model's output. It is separate from `trust=`: a retriever reads your query and returns someone
  else's documents. Unstated, the return value is recorded as `externally-sourced`, and the stage
  lists `"output_trust"` in `assertions.undeclared`. Any other value is refused when the stage is
  decorated. The examples state it on every stage; none of them calls a model, so none shows
  `model-generated`.

The list is written by onetrace, not by your code, so it needs no constant beside it. A stage that
declared no constants of its own carries no `constants` at all:

```json
"assertions": {"undeclared": ["output_trust", "rederivable", "trust"]}
```

A run with undeclared fields is valid and verifies. Nothing is inferred: a field listed in
`undeclared` was not stated by a person, and its recorded value is the cautious default. Add the
meaning when you know it, and the list shrinks.

### 16.3 Which stages a run expects

A decorated run declares its stages up front, as every run does: `"intake"` first when the run
function has parameters, then every `@ot.stage` defined in the run function's module, in the order
they are defined. Pass `@ot.run(stages=[...])` to declare them yourself, for example when the stages
live in several modules.

A stage called during the run that isn't declared is refused at the call. The message names both
fixes: add it to `@ot.run(stages=[...])`, or define the stage in the run's module. Because the
run's arguments are recorded as the `intake` stage, a stage named `"intake"` in the same module as
a run with parameters is refused when it is decorated.

### 16.4 `async def`

Both decorators take `async def` functions, and the decorated function is still a coroutine
function. An async run keeps its record open across every `await`, including in the tasks it
starts with `asyncio.gather` or `asyncio.create_task`: they copy the run with the rest of their
context. Sync and async stages mix freely in one run. The same pipeline written sync and async
records the same nodes, inputs, outputs and edges; only the times, and the order in which
concurrent stages finish, can differ.

### 16.5 A stage called more than once

By default a stage runs once in a run, and its node is the stage's bare name (`retrieve`), as in a
hand-written run. A second call of such a stage is refused with `AmbiguousInstance`, and the message
names both ways to allow it:

- **Declare the stage repeating:** `@ot.stage("refine", ..., repeats=True)`. Every call is
  numbered from the first, `refine#1`, `refine#2`, and so on, even when it runs only once. Loops
  and retries are declared this way.
- **Name each call:** `refine.instance("en")(...)` records that call as `refine#en`. A name given
  this way always wins, in either kind of stage.

A name used twice in one run is refused. So is mixing named and unnamed calls of a stage that runs
once. Calls of one stage that overlap in time (in `asyncio.gather`, or in threads) must each be
named with `.instance()`: numbering them in the order they arrive would not be stable from one run
to the next, so an unnamed overlapping call is refused, even for a repeating stage.

`@ot.run(rederivable=..., note=...)` describe the `intake` stage's instrument, and nothing else.
Left out, the intake records the cautious `"false"` and lists `"rederivable"` in
`assertions.undeclared`. A run function without parameters has no intake, so passing them there is
refused.

### 16.6 Files a stage reads: `files=`

`@ot.stage(..., files=["data/corpus.json"])` digests each named file when the stage starts, before
the function is called, and records it as an input named by its basename (`corpus.json`), with the
stage's trust class. A relative path is read from the working directory at the call.

**This records the file's content when the stage began; it does not prove the function read it.**
If the function changes the file, the record still holds the content it had at the start. Reads
that aren't named in `files=` are not seen.

A missing file is the stage's failure: its receipt records the error, the function isn't called,
and the `FileNotFoundError` reaches you unchanged. Two files with one basename, or a file whose
basename is also a parameter's name, are refused when the stage is decorated, since the two inputs
couldn't be told apart. With `trust="secret"`, nothing is digested and the stage is refused.

### 16.7 When a stage fails

A decorated stage that raises gets a receipt recording the failure exactly as the Recorder API
records it: class `error`, the exception's type as the status, its text as the body, and the
instrument as the origin. The same exception object then reaches your code, unchanged. A stage
that raises `Refusal` records `refused` with its reason and detail, and the call returns `None`. A
return value that can't be encoded fails the stage the same way, with `UnencodableValue`.

The run is still closed and verifies. Each attempt of a retried stage is its own receipt, and all
are kept: declare the stage `repeats=True`, and the failed attempt and the retry are recorded as
`#1` and `#2`.

### 16.8 Switching recording off: `ONETRACE_DISABLE=1`

With `ONETRACE_DISABLE=1` in the environment, `@ot.run` and `@ot.stage` pass every call straight
through: the functions return what they always return, no run folder is created and no file is
written, and none of the refusals that only a recording run makes apply (a second call of a stage,
an undeclared stage, overlapping unnamed calls). `.instance(name)(...)` is a plain call. Nothing
optional is imported because of it.

The variable is read at each call, and only the value `1` switches recording off; unset, empty or
`0`, runs record as usual. Mistakes found when a function is decorated (a lambda as a stage, a run
with no source file and no `manifest=`) are still reported, since they don't depend on recording.

### 16.9 Values of your own types: encoders

Arguments and return values of the types in 16.1 (strings, numbers, booleans, `None`, lists and
dicts of these) are recorded as they are. Two more kinds are recorded without any setup:

- a **dataclass**, as the dict of its fields (a dataclass inside it the same way);
- a **pydantic model**, as `model_dump(mode="json")`. onetrace never imports pydantic itself.

For any other type, register an encoder, a function that turns the value into one of the above:

```python
ot.encoder(Money, lambda m: {"currency": m.currency, "cents": str(m.cents)})
```

An encoder covers subclasses of its type too, unless they have their own, and a registered encoder
wins over the built-in ones. If the encoder raises, or hands back a value of the same type, the
stage stops with `UnencodableValue`, naming the stage, the parameter and the type. A value with no
encoder stops the stage the same way: it is never recorded as `repr()`.

`bytes` passed as an argument are recorded exactly as they are, media type
`application/octet-stream`, the same way a stage's `bytes` return value is stored as `return.bin`.
Bytes inside a list or dict have no JSON form, so they stop the stage; pass them as their own
argument, or register an encoder for the type that holds them.

### 16.10 Choosing the run id: `ONETRACE_RUN_ID`

A decorated run takes its id from `ONETRACE_RUN_ID` when that is set and not empty, and records
`run_id_source: "caller"`; this is how a CI job runs a pipeline and then finds `runs/<id>`.
Otherwise the id is generated, as before. Either way, a `:` in the id is written as `-` in the
folder name only.

The id must be usable as one folder name on Windows and POSIX alike (after `:` becomes `-`): no
`/` or `\`, none of `< > " | ? *`, no control character or line break and no lone surrogate (16.12),
no trailing space or dot, not `.` or `..`, not a name Windows reserves (`CON`, `NUL`, `COM1`, ...),
and at most 255 bytes in UTF-8. Anything else is refused at the call, naming the variable, and
nothing is written.

A run folder that already holds anything is refused at the call, before it is touched: a record is
never merged into or overwritten. That includes a second decorated run in the same process with the
variable still set. Unset the variable or set a new id, or remove the old run. An empty folder is
used as it is. Under `ONETRACE_DISABLE=1` nothing is written, whatever the variable says.

**Its limits:**
- **One id names one run.** A process that calls a decorated run more than once needs a new id for
  each call; with the variable unchanged, the second call is refused.
- **`a:b` and `a-b` name the same folder.** The `:` becomes `-` in the folder name only, so the
  second of the two is refused as a folder that already holds a run.
- **Only `@ot.run` reads it.** A run opened with `Recorder(out_dir, ...)` is written where `out_dir`
  says, whatever the variable holds.

### 16.11 Stages in other threads

A decorated run is found through the calling context, and a thread pool's threads start with an
empty one. So `pool.submit(stage, x)` or `pool.map(stage, xs)` inside a run would find no run and,
as a plain function, record nothing. That is refused instead: a decorated stage called in a thread
with no run, while a decorated run is active anywhere in the process, raises `EmissionRefused`
naming the stage. Carry the run into the thread, with one of these (since 0.4.0):

```python
# one call per item, each named by the item's position (embed#0, embed#1, ...) or by key=
vectors = ot.map(embed, docs, key=lambda d: d.id, workers=8)

# the same for an async def stage, as asyncio tasks
vectors = await ot.gather(aembed, docs, key=lambda d: d.id)

# a pool whose submit carries the run; it names nothing by itself
with ot.pool(workers=8) as pool:
    hit = pool.submit(retrieve.instance("en"), query).result()

# a thread you start yourself
threading.Thread(target=ot.carry(work)).start()
```

`ot.map` and `ot.gather` name each call before any runs: by the item's position in the list you
passed, never the order calls finish, so the same list gives the same nodes in every run. Key by an
id when the list's order isn't fixed: inserting one item at the front of a positional fan-out
changes every node after it, and the comparison says what it sees. A key is a string or an int; two
items with the same key are refused before any call. Results come back in your list's order. A call
that raises is recorded with its error, the others run to the end, and then the first error (in
list order) is raised, with the others named on it. Nothing is lost because a sibling failed.

A carried call belongs to the run it was carried from, even when it runs while another run is
active; one that runs after that run has closed is refused (`run-closed`) and the finished record is
left as it was: join your threads, or leave the pool's `with` block, before the run function returns.

`pool.submit(contextvars.copy_context().run, fn, ...)` still works, and `asyncio.to_thread` copies
the context itself, so it needs nothing. If a call in another thread really belongs to no run, call
the undecorated function, `retrieve.__wrapped__(query)`. With no decorated run active anywhere in the
process, a stage in any thread is a plain function, as outside a run.

**Child processes.** One process writes a run folder. A stage called in a forked child of the run's
process (`multiprocessing` on POSIX, a process pool) is refused by name (`child-process`). A child
started fresh (the `spawn` method, the default on Windows and macOS) carries no run at all, and a
stage there runs unrecorded, as outside any run: do the child's work undecorated and record its
result in a stage of the parent.

### 16.12 Names that become folders

A stage's name and instance name the folder its outputs are staged in, and a decorated run's id
names its run folder. Each must be usable as one folder name on Windows and on POSIX alike,
whichever system records the run, because a run recorded on one is verified and reproduced on the
other. A name that can't be is refused before anything is written, with a `fix:` line naming it:
a stage name when the stage is decorated or the run is opened, an instance when `.instance(name)`
is called or the node starts. Refused: an empty name, `.` and `..`, `/` or `\`, any of
`< > : " | ? *`, a trailing space or dot, a name Windows reserves (`CON`,
`PRN`, `AUX`, `NUL`, `COM1`-`COM9`, `LPT1`-`LPT9`, with or without an extension), and more than
255 bytes. A stage name, an instance and a run id are also refused if they contain a control
character or a line break of any kind (Unicode categories Cc, Cf, Zl and Zp, among them a
right-to-left override or a zero-width space) or a lone surrogate: every name has to print as
itself, in the verifiers' output and the reports. Windows also limits a whole path to 260 characters unless long paths are enabled, which
a name check can't see: keep run folders and names short there.

### 16.13 Linking a query run to its ingest run

Ingest (load, split, embed, index) usually runs once, and query runs read what it built. Say which
recorded ingest run a query run read, on the stages that read it:

```python
@ot.run(corpus=ot.corpus_from("runs/2026-10-01T09-00-00Z-ingest", stages=["retrieve"],
                              index_stage="split"))
def query(question): ...
```

- The first argument is the ingest run's folder, or its chain head (`sha256:` and 64 lower-case
  hex), looked up one level down in the folder that holds your query runs.
- `stages` is required and names stages of this run. A name the run doesn't have, or `"intake"`, is
  refused: when the run is decorated if it states `stages=[...]`, otherwise when it starts, before
  anything is written.
- When the run starts, the ingest run is verified, as `onetrace diff` verifies a run. If it doesn't
  verify, or the chain head isn't found, the query run is refused before anything is written: it
  never links to a record that hasn't been checked.
- `index_stage`, when you give it, names the ingest stage whose output is the chunk index. That
  stage must exist and have one, or the run is refused naming which is missing.
  It is recorded on the linked stages as the constant `corpus_index_stage`, only when given, so a
  change of index stage between two runs shows as a changed setting.
- The link, the ingest run's chain head, is recorded as `assertions.corpus_manifest` on exactly the
  named stages, on every call, and on no other. It needs no constant; a stage whose only assertion
  is the link has no `constants` in its record. A `corpus_manifest` your own code writes with
  `ctx.corpus_manifest` still needs one, and its digest must be in the record's form.

**The link shows which recorded corpus the query run declared it used; it does not prove the
retriever read only that corpus.**

`onetrace localize QUERY_A QUERY_B --doc <document> --runs-root runs/` follows the link one hop
when a query run has no chunk index of its own. It finds the ingest run by its chain head one level
down in `runs/`, verifies it, and takes the chunk index from the ingest stage named as the index.
The hits still come from the query run's own retrieval output, and the view says which run each
part came from. If the ingest run isn't found or doesn't verify, or no index stage is named, the
view reads COULD NOT CHECK and says which.

### 16.14 Settings known only at run time: `ot.constant` and `ot.assertion`

A setting read from a file or an argument can't go in `ot.pkg(config=...)` when the stage is
decorated. Record it from inside the stage instead:

```python
@ot.stage("split", instrument=ot.pkg("splitter", "llama-index-core", kind="transformer"))
def split(docs, settings):
    ot.constant("chunk_size", settings.chunk_size)
    chunks = ...
    ot.assertion("chunk_count", len(chunks))
    return chunks
```

They write the running stage's constants and assertions, numbers and booleans spelled as in
`Instrument.config` (`512` as `"512"`, `False` as `"false"`). An assertion still needs a constant
beside it. Outside a run, and with `ONETRACE_DISABLE=1`, they do nothing. Inside a run but outside
any stage there is nothing to record them on, and they are refused.

**`ot.constant` can be called after a model call.** A stage's constants are written when the stage
returns, so a call anywhere in its body counts, including after the work. Record a setting the
moment you know it, even if only the model's reply tells you (the model version it answered with,
say):

```python
@ot.stage("answer", instrument=llm, output_trust="model-generated")
def answer(prompt):
    reply = client.generate(prompt, temperature=0.2)
    ot.constant("temperature", 0.2)
    ot.constant("model", reply.model)          # known only from the reply
    return reply.text
```

A second call with the same key replaces the first value: the receipt holds the last one. After the
stage has returned, there is no stage to record on, and the call is refused as above.

### 16.15 numpy arrays, LangChain documents, LlamaIndex nodes, and index objects

- **A numpy array** passed to or returned by a stage is recorded as bytes, not as metadata: NumPy's
  own `.npy` form, which carries the dtype, the shape and the order, written from a C-ordered copy,
  media type `application/x-npy`. A returned array is stored as `return.npy`. An array of Python
  objects has no byte form and stops the stage; so does an array inside a list or dict (pass or
  return it on its own). A numpy number inside a value (`np.int64`, `np.float64`, `np.bool_`) is
  recorded as the Python number it holds.
- **LangChain's `Document` and LlamaIndex's nodes** are pydantic models, so they are recorded as
  their JSON dump, with no setup.
- **Index objects** (a FAISS index, a vector-store client) have no encoder, and passing one stops
  the stage. Pass the path of the persisted index instead, and name it in `files=`.

None of these packages is imported by onetrace itself; an encoder is looked up only for a value
whose package you have already imported.

- **LangChain messages** (`AIMessage` and the rest) are pydantic models too, and their dump includes
  `id`, a per-run value (`lc_run--…`): two identical runs then differ at that stage. Register an
  encoder that leaves it out, `ot.encoder(BaseMessage, lambda m: {"type": m.type, "content": m.content,
  "tool_calls": getattr(m, "tool_calls", [])})`; `onetrace doctor` names a recorded output that holds one.

**A tool in your own repository.** `ot.pkg(id, distribution)` reads the version from an installed
distribution. With none installed (code run from its source tree, or vendored), it reads the
`__version__` that the imported module of that name states (since 0.4.0: import it first), and
refuses when there is neither. For code with no version of its own, name the instrument yourself:
`Instrument(id, kind, version, config)` (section 5.2), with the version you pin it by.

### 16.16 Numbers and booleans: rule M for settings, rule V1 for values

A decorated pipeline records numbers and booleans in two places, each with its own rule.

**Rule M: settings and assertions** (`ot.constant`, `ot.assertion`, and an instrument's `config`
when a stage records it: `ot.pkg(config={"top_k": 4})` and `config={"top_k": "4"}` record the same
digest).
A record's constants and assertions are strings, so a value is written as its string:

- a `bool` as `"true"` or `"false"`;
- an `int` as its decimal string (`3` as `"3"`);
- a `float` in RFC 8785's number form, the shortest decimal that reads back as the same double:
  `0.1 + 0.2` as `"0.30000000000000004"`, `1e21` as `"1e+21"`, `-0.0` as `"0"`. Exponent form is
  used outside 10^-6 <= |x| < 10^21;
- NaN and Infinity have no such form, and are refused, naming the key.

No type is recorded beside the value: `0.7` and `"0.7"` give the same digest, as `3` and `"3"` do.
If the difference matters to you, say it in the key or the value.

**Rule V1: arguments and return values** recorded as artifact bytes (16.1). These are stored as
RFC 8785 JSON, media type `application/json`:

- numbers are allowed: floats in rule M's form, integers within +-(2^53 - 1); a larger integer
  stops the stage with `UnencodableValue`, whose fix is to record it as a string;
- booleans are JSON `true` and `false`;
- keys are sorted and there is no whitespace.

**The two encodings, named apart:**

- **`ctx.write_json`: the SDK's canonical form, no numbers.** The Recorder API's own form is
  unchanged: it refuses a number, so a value you write yourself is always a string you chose.
- **Decorated values: RFC 8785, numbers allowed.** Only what the decorators store for you
  (arguments and return values) uses it.

Neither changes the record's format: a receipt records an output's name, digest and media type,
not how its bytes were encoded.

### 16.17 `@ot.run`'s defaults

Everything `@ot.run` doesn't take from its arguments, it takes from these defaults:

| Option | Default |
|---|---|
| `policy` | `"fail-closed"` (5.8) |
| `manifest` | the source file of the module that defines the run function. With no source file (a REPL, a notebook cell) `@ot.run` is refused when it decorates, with `fix: pass manifest=<path to your pipeline file>` |
| topology | always on for a decorated run: its manifest carries `edges`, since a decorated pipeline may fan out, fan in or overlap, which a run in declared order can't record |
| `stages` | optional. Left out: `"intake"` (when the run function has parameters), then every `@ot.stage` name defined in the run function's module, in definition order, each once (16.3) |
| the intake stage | `"intake"`, only when the run function has parameters: each argument is recorded as an input named by its parameter, in rule V1's encoding (16.16), with the run's `trust=` (16.2). A stage of your own named `"intake"` in that module is refused |
| the return value | not recorded on its own: what a run returns is a stage's return value, recorded by that stage (16.1) |
| `run_id` | generated, recorded with `run_id_source: "generated"`; `ONETRACE_RUN_ID` gives your own (16.10) |
| `run_dir` | `"runs/{run_id}"`, with a `:` in the id written as `-` in the folder name only; the recorded `run_id` keeps its `:` |
| `sign_with` | none: the run isn't signed. A key file's path, or `onetrace.env("NAME")`, signs the run at close (17.2) |
| everything else | the `Recorder`'s own defaults (5.1) |

### 16.18 What a run says when it closes: `ONETRACE_QUIET=1`

When a decorated run closes, `@ot.run` writes two lines to stderr: the folder the run was written to,
and the `onetrace-verify` command that verifies it.

```
onetrace: run written to runs/demo
next: onetrace-verify --require-artifacts runs/demo
```

They are written once per run, after the run's record is complete, also when a stage raised and the
run ends in an error. A folder holding a space, or a character a shell would read, is shown in double
quotes, so the command can be pasted as it is; the folder is shown with `/` on every system, which PowerShell,
cmd.exe and sh all accept. stdout is never written to.

Only a decorated run (`@ot.run`) says this. A `Recorder` you open and close yourself (section 5)
writes nothing to stderr when it closes.

When nothing was recorded inside the run (its stages ran in threads or tasks that don't carry the
run's context, 16.11), there is no record, and the first line says so instead:
`onetrace: no run written to runs/demo: no stage was recorded inside the run`, with how to call the
stages inside it.

`ONETRACE_QUIET=1` in the environment suppresses the two lines; only the value `1` does. With
`ONETRACE_DISABLE=1` (16.8) no run is opened, so nothing is written.

The same setting also silences `onetrace-verify`'s summary line and its `fix` lines, which it writes to stderr (section 8.1). Its rows and verdict on stdout, its exit code and `--json` don't change.

### 16.19 Keeping a stage's text inputs: `keep_inputs`

A stage's inputs are recorded as digests, never as text. To read what a stage was given, keep the text of named inputs beside their digests:

```python
@ot.stage("answer", instrument=llm, keep_inputs=["question"])
def answer(question, passages):
    ...
```

**A stage with `keep_inputs` stores that input's text in plain form in `inputs/`. Keep inputs only when you would also keep them in a log. `trust="secret"` inputs are never kept.** It is off unless you name an input, and a person decides it, as for `trust=`.

- **What is kept.** Each named input whose value is a `str`, stored in `inputs/<receipt stem>/<name>.json` as exactly the bytes its digest was computed over: the text as a JSON string. The file's SHA-256 is the input's digest, and the receipt is unchanged: the same input entry, the same digest. `inputs/` is beside `artifacts/`, never inside it. `@ot.run(keep_inputs=[...])` does the same for the run's own arguments.
- **What is not kept, and the stage goes on:** a value that is not text, recorded as before and listed as `not text (<type>)`; an input whose stored bytes are over the cap, listed as `over <cap> bytes`; an input not passed, `no value passed`; an earlier stage's output passed on whole, which is a link to that stage, `an earlier stage's output`. A stage that doesn't finish keeps nothing: its files are removed, and its inputs are listed as `the stage did not finish (<outcome>)`.
- **The cap is on the stored bytes,** the JSON string as written to the file, so the cap and the file's size are the same number: `keep_inputs_cap=`, 4096 by default, at most 65536. A name that is not a parameter of the function, or a cap out of range, is refused when the stage is decorated.
- **The record says what was kept,** in two constants only onetrace writes: `onetrace.kept_inputs` lists the kept inputs, and `onetrace.inputs_not_kept` each named input that wasn't, with why. A constant of your own whose name starts with `onetrace.` is refused, whether set with `ctx.constant` or straight into `ctx.constants`; it is read once, as 0.3.0 read it (falsy as no constants, anything else as `dict()` reads it), and a name that is a `str` subclass (a `StrEnum` member, say) is read as its characters, as 0.3.0 recorded it. This guards against mistakes through the public interface: a stage's code runs in the recorder's own process and could still write anything there, so a record is only as trustworthy as the code that ran beside it. Kept text counts towards `max_run_bytes`.
- **A reader shows the text only when the record lists the input as kept and the file's SHA-256 equals its digest.** `onetrace diff --text` shows a changed kept input as `- "<baseline>"` and `+ "<candidate>"` under its line, each escaped one character for one character as the verifier escapes names. Otherwise it reads `not kept`, `not kept: file missing`, or `text does not match its digest; not shown`. A file under `inputs/` that the record does not list is never shown.
- **Checking it.** `onetrace-verify` (0.2.0 and 0.2.1) does not read `inputs/`: its result is the same with or without the files. The SDK's own check (`python -m onetrace.sdk_verify`) adds a row per kept input, `kept input text matches its digest`: PASS, FAIL, or NOT RUN when the file was removed; and a note, `inputs/: unlisted file`, for a file no receipt lists.
- **Removing the text** (all of `inputs/`, or one file) is allowed and leaves the run verifiable: the digests are in the receipts.

### 16.20 The model-call record: `onetrace[model-call]`

`onetrace.model_call` writes one canonical **model-call record** per model call, as an ordinary output of the stage that made it. It never changes the receipt format and never changes the reading of a run's receipts; its own verifier rows read as every other row does, and a FAIL on one fails the run like any other FAIL. A run that writes no record is byte-for-byte the run it would be without the feature. Installed with `onetrace[model-call]`; the vLLM and transformers adapters import the engine only when used, so the engine stays your own version.

```python
import onetrace.model_call as mc
from onetrace.model_call import hosted, record

def answer(ctx, question):
    rec, status = hosted.call(transport, endpoint, request, key, ctx=ctx)   # writes model_call.json
    return mc.load(rec)
```

Every field that describes the call is a value object in one of five states — `observed`, `declared, not observed`, `declared, contradicted`, `not observed`, `not applicable` — with its source; a field the writer cannot fill is `not observed` or `not applicable`, never left out. The first record a stage writes is `model_call.json`, then `model_call.2.json` and so on. A per-step capture, when taken, is a separate `model-generated` output (`model_call.<n>.steps.json`) the record names by digest; the record holds no token id or text at any level. `mc.compare(a, b)` returns `same`, `diverged` or `could not compare`, and nothing else. `ot.model_from(run, stages=[...], serve_stage=...)` passed as `@ot.run(model=...)` records a verified model link in the corpus-link form. A verifier gains record rows for a run that holds a record, reading as its receipt rows do. The record, its states and sources, the capture levels, the comparator's results, the verifier's rows and what the record does not claim are specified in [the model-call record](model-call-record.md), with its schema `docs/model_call_record.schema.json` (`model-call-record/1.0`).

## 17. Signing a run

A signature is an Ed25519 signature over a finished run's chain head, by a key you hold. It is stored as its own file, `signatures/<n>.json`, beside the run's receipts. Like an anchor, it is never part of the chain: signing changes no byte that was there before, so the chain head stays byte-identical, and a run may carry several signatures (a recorder and a reviewer, say). Signing and anchoring work together in either order.

**A signature does not show:**
- that the key wasn't stolen or misused;
- that the run's output is correct;
- when the record was signed (that is an anchor's job);
- who the person behind the key is. The verifier knows keys, not people; your trust list is how you connect a key to a name.

What it does show: the holder of that private key signed this exact record.

### 17.1 Making a key

On macOS or Linux:

```
mkdir -p ~/keys
onetrace keygen --out ~/keys/recorder.key
```

On Windows, in PowerShell:

```
New-Item -ItemType Directory -Force "$HOME\keys"
onetrace keygen --out "$HOME\keys\recorder.key"
```

On Windows, a folder in your user profile (`$HOME`, which is `C:\Users\<you>`) is private to you by default: other standard users on the machine can't read it, but administrators can. Step 8 of [Get started](https://oneproof.dev/get-started.html) makes a key and signs a run, start to finish.

`onetrace keygen` writes a new private key as PEM, with owner-only permissions (0600) on Linux and macOS, and prints the public key, its `key_id` (the `sha256:` fingerprint of the public key's 32 raw bytes) and where the private key was written; it never prints the private key. It never overwrites a file, and it refuses a path inside a run folder. It doesn't create folders, so make the key's folder first, as above: a path whose folder is missing is refused, and nothing is written. On Windows it can't set owner-only permissions itself: the file inherits its folder's access list, and `keygen` says so. Keep the key in a folder only you can read.

`--test-only` adds a first line saying the key was made for tests. A test key is made for one test and never reused.

### 17.2 Signing

- **At close:** `Recorder(..., sign_with="~/keys/recorder.key")`, or `sign_with=onetrace.env("ONETRACE_SIGNING_KEY")` for a key held in an environment variable as PEM text. `@ot.run(sign_with=...)` passes it to its recorder. A key path may start with `~`. The key is checked before the run writes anything; the run is signed after its manifest is written and verified, once.
  - A run that recorded nothing writes no manifest, so there is nothing to sign: `close()` returns as it would without `sign_with`.
  - If signing fails at close (the key file was removed since the run started, say), `close()` raises `SignRefused`: the manifest stands, the run is unsigned, and the run folder is released. Sign it afterwards with `onetrace sign`. If the signature was stored but couldn't be confirmed, `close()` raises `SignNotConfirmed`, and a second `close()` doesn't sign again. Under `@ot.run` either exception comes out of the decorated call.
  - A signature says who signed the record, not that it verified: under `fail-open`, a run the verifier refused is still signed. Check it with `onetrace-verify`.
- **Afterwards:** `onetrace sign RUN --key PATH` or `onetrace sign RUN --key-env NAME`, with an optional `--signer "label"`. It exits 0 when the signature is stored and confirmed, 1 when it is refused (nothing stored), and 3 when it was stored but could not be confirmed as landed in this run.

**Key custody.** The key is never written into a run folder, never logged and never printed. A key file inside the run folder is refused before it is read, whether it is really there, reached through a link inside the folder, or a hard link to it anywhere under the folder (the refusal names that file, never the key). A key path must be a regular file of at most 64 KiB. In CI, keep the key in the CI secret store and pass it as an environment variable with `--key-env` or `onetrace.env(...)`.

`--signer` (or `signer=`) is a label the signer claims, such as "nightly CI". It is stored as a claim and only ever reported as claimed: the name the verifier shows as the signer comes from your trust list, never from the record.

Signatures are stored the way anchors are (section 15): names are allocated without overwriting, the record is linked in from beside `signatures/`, and it is reported stored only when it is confirmed in this run's folder. **Sign a run folder that nobody else is writing to.**

### 17.3 Checking signatures: `onetrace-verify RUN --trust FILE`

The trust file lists the keys you trust, each with your label for it:

```json
{"format": "onetrace-trust/0.1", "keys": [{"key_id": "sha256:<64 hex>", "label": "ci-key"}]}
```

Each file in `signatures/` gets one row, after the anchors' rows:

| Signature | Row |
|---|---|
| Valid, and the key is on your trust list | **PASS**: "signed by ci-key (sha256:…)" |
| Valid, but the key is not on your trust list (or no `--trust` was given) | **SIGNED-UNTRUSTED**: "signed by an untrusted key sha256:… -- signer not on the reader's trust list". Never a pass. |
| An `alg` this verifier doesn't implement | **NOT-RUN**, naming the algorithm |
| Without `onetrace-verify[crypto]` | **NOT-RUN**: "install onetrace-verify[crypto] to check signatures" |
| Invalid, or for another run's chain head, or malformed | **FAIL** |

A FAIL makes the verifier's result FAIL (exit 1). SIGNED-UNTRUSTED and NOT-RUN leave the result as it would be without the signature. After the summary, each count has its own line. An untrusted signature reads "records beside the chain: 1 signed by an untrusted key -- signed, and the signature is valid, but by a key not in your trust file: it shows the record is unchanged since signing, not who signed it. It leaves the result as it would be without it." `--json` gives the counts as `summary.records_signed_untrusted` and `summary.records_not_run`, each with its note. For a signed run, the text output also prints what a signature does not show, and `--json` carries the same four statements as `"non_claims"`. A run with no signature reads exactly as before. The command a run prints when it closes (`next: onetrace-verify --require-artifacts …`, section 16.18) has no `--trust`, since a run can't know its reader's trust file: add `--trust FILE` to see a trusted signature as PASS.

**The order of the checks is fixed.** The file's size and nesting caps, the record's structure and its binding to this run's chain head come first, with or without the extra and whatever the algorithm; a failure there is FAIL. Only then may a row read NOT-RUN. A malformed record, or one for another run, is never shown as NOT-RUN.

**The record:** `format`, `"stage-receipt-signature/0.1"`; `chain_head`; `alg`, `"ed25519"`; `public_key`, 64 lowercase hex; `key_id`, which must be the public key's fingerprint; `signature`, base64 of the 64-byte signature over `stage-receipt-signature/0.1:` followed by the chain head; and optionally `signer`, the claimed label. A record written as `onetrace-signature/0.1`, the name before 0.2.0, is FAIL: a development build before 0.2.0 wrote it, and it can't be made valid. The fix is to move it out of `signatures/` and sign the run again with `onetrace sign`, which writes the new one.

**onetrace's own strictness.** The approved −01 text leaves these open:
- a member the record format doesn't define is FAIL, and the row names it;
- a file in `signatures/` that isn't named `<n>.json` is FAIL;
- a `signatures/` that is a link is FAIL, and nothing in it is read.

`diff` and `localize` never change a verdict for a signature. A signed run's report carries its rows under `"signing"`, as the verifier reads them with no trust list, so a valid signature shows as SIGNED-UNTRUSTED there.

## 18. Seeing a run in your tracing tool

**A trace is a view of a record, never evidence.** Collectors drop, sample, rewrite and re-time
spans. Whether two runs agree, or whether a run is intact, is answered by `onetrace-verify` and the
comparison verbs reading the record, never by a trace. Nothing in onetrace reads a trace back.

```sh
onetrace export otel runs/<run id> --out trace.json
```

writes the run as one OpenTelemetry trace in OTLP/JSON. It makes no network calls and needs no
OpenTelemetry package. Sending the file to your collector is your step, with the collector's own
file receiver. `--service-name NAME` sets the resource's `service.name` (default `onetrace`). A run
that can't be read is refused (exit 2) and nothing is written.

What it writes:

| onetrace | the trace |
|---|---|
| the run | one trace, a root span `onetrace.run` from the first stage's start to the last stage's end |
| trace id | the first 16 bytes of the chain head |
| a receipt | a span `onetrace.stage <node>`, a child of the root; its id is the first 8 bytes of the receipt file's sha256 |
| a `receipt:` input | a span link, `onetrace.edge = "receipt"`; with `link_parts=True`, one per linked parameter, with `onetrace.param` and the pointer's depth and digest (`onetrace.pointer.depth`, `onetrace.pointer.digest`), never the pointer: its keys are part of the output (a digest can confirm a key someone guesses) |
| a held step's attempted link | a span link, `onetrace.edge = "attempted"` |
| outcome ok | status Unset |
| outcome error | status Error, `error: <exception type> (origin <origin>)`; the exception's text is not exported |
| outcome refused | status Error, `refused: <reason>`; the root span is Error too |
| the run id, chain head, coverage, anchor state, `outputs_kept` | attributes on the root span (`onetrace.run_id`, `onetrace.chain_head`, `onetrace.coverage.*`, `onetrace.anchor.state`, `onetrace.outputs_kept`), with `onetrace.note`: a view of the record, never evidence |
| the SDK and the record's format | the resource's `onetrace.sdk.version` and `onetrace.format`; the scope's `name` and `version` |

A stage span also carries its stage and node, `onetrace.outcome`, its start and end from the
receipt's `time`, its receipt digest, its instrument's name, version and
digest, the counts and digests of its inputs and outputs (at most 64 digests each; when cut,
`onetrace.inputs.digests_cut = "true"`), its output bytes, and a count per trust class.

**Never exported:** input, output, constant or assertion values (a part link's pointer included), an
exception's text, file paths. Names are exported: a stage's, a node's, a parameter's. A key you give
`ot.map` becomes part of a node's name, so choose one that isn't a value you wouldn't send.

**The export doesn't verify.** A run onetrace-verify fails still exports, and nothing in the trace
says so. Verify the run first.
A trace backend is a wider audience than the run folder. To put a value on a span, add it with your
own tracer.

The same run always exports to the same bytes. A changed receipt changes its own span's id and the
id of every span after it, since each later receipt's `prev` link changes with it.

<!-- https://oneproof.dev/onetrace/errors.md -->

# onetrace errors

Every refusal from onetrace ends with a `fix:` line and a `see:` line naming one of the
sections below. Each section says what the refusal means and what to do about it.

This page is generated from `onetrace/errors.py`; edit the table there, not this page.

## run-declaration

The run was opened without something every record needs: a run id, a non-empty list of distinct declared stages, or an emission policy (`fail-closed` or `fail-open`).

**Fix:** Give the run its id, its declared stages and its policy. With `@ot.run`, name the stages in `stages=[...]`.

## run-bytes-cap

`max_run_bytes` is smaller than the smallest receipt this run's own declared stages could have, so a stage that crossed it couldn't even be given a receipt saying why.

**Fix:** Set `max_run_bytes` to at least the minimum the message names, or leave it unset.

## run-folder-used

The run folder already holds a run. A record is never overwritten or merged into.

**Fix:** Give the run a new run id or run folder, or move the old run away first.

## run-folder-locked

Another process holds the run folder: its lock file is present. A run is written by one process.

**Fix:** Wait for the other process to finish, or give this run its own folder. Remove a lock file only when you know no process is writing that run.

## run-manifest

`@ot.run` could not find the source file to record as the run's manifest.

**Fix:** Pass `manifest=<path to your pipeline file>` to `@ot.run`.

## folder-name

A stage name, an instance or a run id names a folder in the run, and this one can't: it is empty, reserved, too long, or holds a character a file system refuses.

**Fix:** Rename it, using letters, digits, spaces, `-`, `_` and `.`.

## intake-name

A stage uses the name the run records its own arguments under (the intake stage).

**Fix:** Rename that stage.

## no-stage-context

Something that belongs to a stage (`ot.constant`, `ot.assertion`, the stage context) was used where no stage is running.

**Fix:** Call it inside the decorated stage whose output it describes.

## decision

A step cited an onedoor decision with `ctx.decision`, but what it passed is not a decision-reference object, or the permit it passed is not a compact JWS string.

**Fix:** Pass the `onedoor-decision-ref/1` object as `ref`; pass the bound permit's compact JWS (a string) as `permit` where one was issued; pass the action object the step is about to send as `action`.

## verify-nested

A nested run does not match the head the outer run recorded for it: the inner run's own chain head differs from what the outer run recorded as that run's input, or an inner run's folder was given but its manifest cannot be read.

**Fix:** Give each nested run's folder with --nested-run; restore the outer and inner records as they were written, so the inner run's chain head equals the head the outer run recorded.

## nesting-depth

A stage was called more deeply nested than the limit onetrace records (NEST_DEPTH_LIMIT). A call past the limit is refused and named, never recorded as a silent gap, so a runaway recursion cannot fill a run folder.

**Fix:** Reduce the nesting depth, or flatten the recursion into declared stages; the limit is stated in docs/nesting.md.

## traceparent

The `TRACEPARENT` environment variable is set to something that is not a W3C traceparent, so the run or a step could not record the trace context it ran in. A trace is a view, not evidence, and a broken one is never recorded.

**Fix:** Set TRACEPARENT to `00-<32 hex trace id>-<16 hex span id>-<2 hex flags>` (neither id all zero), or unset it; a run with no active span and no TRACEPARENT records no trace context.

## keep-inputs

`keep_inputs` names something that is not a parameter of the function, or `keep_inputs_cap` is not a whole number of bytes from 1 to 65536. Checked when the stage is decorated.

**Fix:** Name the function's own parameters, spelled as in its signature, and give the cap in bytes (default 4096).

## reserved-constant

A constant's name starts with `onetrace.`, which only onetrace's own constants use (`onetrace.kept_inputs`, `onetrace.inputs_not_kept`, `onetrace.links`), whether it was set with `ctx.constant` or straight into `ctx.constants`.

**Fix:** Rename the constant without the `onetrace.` prefix.

## constant-name

A stage's constants can't be read as a dict, a constant's name is not a str, or two names read as the same characters (a `str` subclass, such as a `StrEnum` member, is read as its characters; the constants themselves as `dict()` reads them, as 0.3.0 did).

**Fix:** Add constants with `ctx.constant(name, value)` or by a `str` key in `ctx.constants`, and give each constant its own name.

## stage-undeclared

A stage ran, or was named, that the run doesn't declare.

**Fix:** Add it to the run's declared stages, `@ot.run(stages=[..., 'name'])`, or call the undecorated function instead.

## stage-order

A stage ran out of the declared order. In a linear run, each stage is the next declared one.

**Fix:** Call the stages in the order the run declares them, or declare the order they really run in.

## node-repeated

A node has already written its receipt. Each node id is written once in a run.

**Fix:** Name each call of a stage that runs more than once (`.instance(name)`), or declare it `repeats=True`.

## repeated-call

A stage was called again, or by overlapping calls, in a way that wouldn't name each call the same way in every run.

**Fix:** Declare the stage `repeats=True`, which numbers its calls, or name each call with `fn.instance(name)(...)`; give overlapping calls names of their own.

## instance-name

An instance name is empty or not a string.

**Fix:** Name the instance with a non-empty string: a branch name such as `a`, or an iteration number such as `1`.

## no-run-in-thread

A decorated stage was called in a thread that doesn't carry the active run. A stage is never left out of a run silently.

**Fix:** Carry the run into the thread: `ot.pool()` for a pool, `ot.map(stage, items)` for a fan-out, `ot.carry(fn)` for a thread you start (or `pool.submit(contextvars.copy_context().run, fn, ...)`); or, when the call belongs to no run, call `fn.__wrapped__(...)`.

## fan-out-key

`ot.map` or `ot.gather` was given something that isn't a decorated stage, an item whose key is not a string or an int, or two items with the same key.

**Fix:** Pass a function decorated with `@ot.stage`, and a `key=` that gives each item its own string or int (a document id, say); left out, the key is the item's position in the list.

## run-closed

A stage was called after its run closed: a call carried into the run (`ot.carry`, `ot.pool`, a copied context) that ran after the run function returned. A finished record is never added to.

**Fix:** Wait for the carried calls before the run function returns (join the thread, leave the pool's `with` block), or call the stage inside a run that is still open.

## export-unreadable

`onetrace export otel` could not read the run well enough to export it: a manifest or receipt that doesn't parse, a chain entry outside the run folder, a chain head or time it can't read. Nothing is written; a partial trace is never exported.

**Fix:** Run `onetrace-verify` on the run: it names what is wrong with the record. Export a run it reads.

## child-process

A decorated stage was called in a child process (a fork of the run's process, as `multiprocessing` and process pools make). One process writes a run folder.

**Fix:** Call the stage in the run's own process, in threads (`ot.map`, `ot.pool`) or asyncio tasks (`ot.gather`); run the child's work undecorated (`fn.__wrapped__`) and record its result in a stage of the parent.

## stage-function

`@ot.stage` was put on something with no single return value to record: a generator, an async generator, or a lambda.

**Fix:** Decorate a plain or async function that returns its result.

## meaning-fields

A meaning field was given without the one it explains: `note=` explains a stated `rederivable=`; or `rederivable=` and `note=` were given to a run function that has no arguments to describe.

**Fix:** Pass `rederivable=` with `note=`, or drop `note=`. Meaning fields are a person's answer: leave them out rather than guess.

## clock-backwards

A stage's end time came before its start time: the clock went backwards during the stage.

**Fix:** Run on a clock that doesn't step backwards (a synchronised system clock), and run the stage again.

## value-unencodable

A value a stage takes or returns, or a constant or assertion, has no form in the record's JSON: NaN or Infinity, an integer beyond 2^53 - 1, a non-string dict key, bytes or an array inside a list or dict, or a type with no encoder.

**Fix:** Record it in a form JSON keeps (a string, for example), pass bytes and arrays as their own argument, or register an encoder with `ot.encoder(Type, fn)`.

## encoder

`ot.encoder` was given something other than a type and a callable, or an encoder raised, or returned the same type it was given.

**Fix:** Register `ot.encoder(MyType, fn)` with a function that returns a dict, list or string.

## instrument

A stage's instrument can't be recorded: it has no name, its version doesn't pin a build (empty, `latest`, a branch name), its package isn't installed, `rederivable` isn't `true` or `false`, or its configuration has no canonical form.

**Fix:** Name the instrument and pin its exact version; install its package, or build the Instrument yourself.

## instrument-declaration

The run's instruments, declared when it opened, don't match its stages: a stage with none, a stage given a second one, or an instrument for a stage the run doesn't declare.

**Fix:** Declare one Instrument for each declared stage, by stage name or node id, either when the run opens or on each stage, not both.

## trust-class

An input or output was given a trust class that isn't one of `operator-authored`, `model-generated`, `externally-sourced`.

**Fix:** Use one of the three trust classes. Which one is a person's answer: ask rather than guess.

## secret-input

An input is declared secret, and the emitter never computes a digest over a secret.

**Fix:** Leave the secret out of the stage's recorded inputs; record the name of the setting that holds it instead.

## input-unidentified

A stage read an input that has no stored artifact, so nothing identifies what it read.

**Fix:** Read the input through the stage context (`ctx.read_external`), which stores what it read.

## input-changed

An input's bytes no longer match the digest recorded for them: the file changed after it was recorded, or the digest supplied for it is wrong.

**Fix:** Don't change an input while a run reads it. Pass the digest of the bytes as they are, or let the SDK compute it.

## input-name-clash

A file named in `files=` is recorded under a name a parameter already has.

**Fix:** Rename the file or the parameter, or record one of them yourself with `ctx.read_external(path, name=...)`.

## output-name

An output's name isn't one plain file name that every system stores as written: it has a folder part, or it is a name Windows would store under another name.

**Fix:** Give the output one plain file name: letters, digits, spaces, `-`, `_` and `.`, with no folder part.

## output-name-repeated

A stage wrote a second output under a name it had already written. The second write would replace the stored file the receipt already lists, and the record would no longer verify. Different stages, and different instances of one stage, may use the same name.

**Fix:** Give each output of a stage a distinct name.

## receipt-read

A stage asked for another node's receipt that isn't in this run, or named a stage with several instances without saying which.

**Fix:** Name a node that has run, and name the instance when the stage ran more than once.

## json-output

`write_json` was given something it doesn't write: a top-level value that isn't a JSON object, or a number JSON can't keep exactly.

**Fix:** Wrap the value in an object, e.g. `{"items": [...]}`, and turn numbers it can't keep into decimal strings first.

## assertion-constants

An assertion was recorded without the constants it holds under, or `constants` or `undeclared` was used as an assertion's name (with `ctx.assertion` or straight into `ctx.assertions`).

**Fix:** Record the constants with `ot.constant` (or `ctx.constant`) alongside the assertion.

## boundary

A boundary isn't a name and kind (with an optional note), or names a stage the run doesn't declare, so a reader couldn't place it.

**Fix:** Give the boundary a declared stage's name and a kind, each a non-empty string.

## gap

A gap names a stage the run doesn't declare, or gives no reason. A gap records a receipt that was never written, and why.

**Fix:** Name a declared stage and say why its receipt is missing.

## edges

A declared or observed edge doesn't fit the run: not a from/to pair, a stage it doesn't declare, declared twice, `repeats` other than `true`, or an edge to no receipt in the run.

**Fix:** Declare each edge once, between declared stages, and give `repeats` only as `true`.

## receipt-write

A receipt couldn't be written in canonical form or at all. Under `fail-closed` the stage's result is discarded rather than left unrecorded.

**Fix:** Fix the cause the message names (often a full disk or a folder that can't be written), and run again.

## anchor-heads

The anchor block names chain heads that aren't a non-empty list of chain-head strings, or an index outside them.

**Fix:** Pass a non-empty list of chain heads and an index within it.

## chunk-index

`ot.chunks` was given something it can't index: `keep_text` that isn't True or False, an item that isn't a (doc, text) pair, a document that isn't a non-empty name, or a chunk's text that isn't a string.

**Fix:** Pass `ot.chunks` a list of (doc, text) pairs, each doc a non-empty string and each text a str, and `keep_text` as True or False.

## corpus-manifest

A corpus manifest is malformed: its digest isn't `sha256:` and 64 lower-case hex; member, preimage and proof weren't given together; or the stage wrote one as well as the run's corpus link.

**Fix:** Pass the digest as `sha256:` and 64 lower-case hex, give member, preimage and proof together, and leave the link to `ot.corpus_from`.

## corpus-link

`ot.corpus_from` can't link this query run to its ingest run: no run at the place given, a run that doesn't verify, no such index stage, or stages that aren't the run's own.

**Fix:** Pass the ingest run's folder (or its chain head) and the stages of this run that read the corpus; link only a run that verifies.

## model-link

`ot.model_from` can't link this run to its provisioning run: no run at the place given, a run that doesn't verify, no such serve stage, or stages that aren't the run's own.

**Fix:** Pass the provisioning run's folder (or its chain head) and the model steps of this run; link only a run that verifies.

## anchor-run

`onetrace anchor` can't read the run's chain head: there is no run folder at the path given (or it is a file), the run has no MANIFEST.json (it isn't finished), or the manifest can't be read, is over its size or depth cap, or names a chain head of the wrong form.

**Fix:** Anchor a finished run, by its folder, and check that onetrace-verify reads the run as intact first.

## anchor-record

The anchor record isn't one the writer accepts: malformed, or not a record of this run's chain head.

**Fix:** Let `onetrace anchor` build the record from its source; don't write anchors/ by hand.

## anchor-folder

The run's anchors/ folder can't be written safely: it is a link, not a folder, can't be prepared or listed, has no free name, or changed while the record was written. Nothing is stored or claimed.

**Fix:** Anchor a local copy of the run in a folder nobody else is writing to, and run the command again.

## anchor-tsa

The RFC 3161 time-stamp authority's answer can't be used: the [crypto] extra is missing, the authority can't be reached or refused, or its response is too large, of the wrong form, for another request, or doesn't verify against the certificate and root you pinned.

**Fix:** Install `onetrace-verify[crypto]`, check `--tsa-url`, and pin the authority's own certificate and root with `--tsa-cert` and `--tsa-root`.

## anchor-calendar

No OpenTimestamps calendar gave a usable answer: none named, one not https, none answered, or every proof still pending.

**Fix:** Name each calendar as `--calendar https://...`; a pending proof reaches a Bitcoin block after some hours, so run `--upgrade` again later.

## anchor-headers

An upgraded OpenTimestamps proof can't be checked against your block headers: the block isn't in the file, the file can't be read, or the proof would read FAIL.

**Fix:** Pass `--headers` a JSON object of height to the 80-byte header as 160 lowercase hex, including the block the message names, taken from the chain itself.

## signature-folder

The run's signatures/ folder can't be written safely: it is a link, not a folder, can't be prepared or listed, has no free name, or changed while the record was written. Nothing is stored or claimed.

**Fix:** Sign a local copy of the run in a folder nobody else is writing to, and run the command again.

## signing-key

The signing key can't be used: it isn't a key file's path or `onetrace.env("NAME")`, the variable is unset, the file can't be read, isn't a regular file or is too large, it isn't an unencrypted Ed25519 private key in PEM form, the [crypto] extra is missing, or the key file (or a hard link to it) is inside the run folder. For `onetrace keygen`: the place given for the new key is inside a run folder, already exists, or can't be written.

**Fix:** Keep signing keys outside every run folder, and sign with a key `onetrace keygen` made: its file's path, or `onetrace.env("NAME")` for its PEM text in the environment. For `onetrace keygen`, when the file already exists: choose another path; a key is never overwritten. When it can't be created: create the folder first, or choose a path you can write.

## signing-store

The run can't be signed as asked: there is no run folder at the path given (or it is a file), it isn't finished (it has no MANIFEST.json), the signer label isn't a non-empty string within its length, the record would read FAIL, or its temporary file can't be written. Nothing is stored.

**Fix:** Sign a finished run that onetrace-verify reads as intact, with a short signer label or none, and run the command again.

## verify-receipt

A receipt doesn't read as written: its file, its bytes, its form or a field the format requires. Each FAIL row names the receipt and the check.

**Fix:** Restore the receipt exactly as it was written; a record is never edited. If the recorder wrote it so, record the run again with a recorder that writes the field.

## verify-chain

The chain doesn't hold together: a receipt's link to the one before it, its digest in the manifest, the manifest's own head, or the order of the stages.

**Fix:** Restore the manifest and the receipts exactly as they were written, in their order.

## verify-artifacts

An output file a receipt names is missing from the run's artifacts/ folder, unreadable, or not the file whose digest the receipt records.

**Fix:** Restore the run's artifacts/ folder as it was written, or verify without `--require-artifacts` when the outputs were deliberately not kept.

## verify-model-call

A model-call record an output names fails its check: it doesn't parse or isn't canonical, names a format this verifier doesn't implement, is missing or has a malformed field, holds a value where it is not observed, or carries a string shaped like an authorization header. Each FAIL row names the record and the check.

**Fix:** Restore the record as it was written; a record is never edited. If the writer wrote it so, record the run again with a writer that writes the field, and write no credential into any recorded field.

## verify-anchor

A record beside the chain (an anchor in anchors/) fails its check, or the file you passed to check it (`--headers`, `--tsa-cert`) can't be read.

**Fix:** Restore the record as it was written, or anchor the run again with `onetrace anchor`; pass the header or certificate files in the form `onetrace-verify --help` names.

## verify-signature

A record in signatures/ fails its check: it is malformed, has an unknown member or the format name before 0.2.0, carries a key_id that isn't its key's fingerprint, covers another chain head, or doesn't verify; or signatures/ holds a file that isn't a `<n>.json` record, or is a link. Or the trust file you passed (`--trust`) can't be read.

**Fix:** Sign the run again with `onetrace sign`; a signature is never edited. Pass the trust file in the form `onetrace-verify --help` names. For a record in the format name before 0.2.0, `onetrace-signature/0.1`: move the record out of signatures/ (a development build before 0.2.0 wrote it, and no release writes that format), then sign the run again with onetrace sign.

## verify-plan

The run's edges don't match its approved plan (`declared_edges`), or the manifest's edges aren't the ones its receipts record.

**Fix:** Record the run again on the approved plan, or approve the plan it ran; restore the manifest if it was edited.

## verify-decision

An onedoor decision row fails: a declared gated step cited no decision, a reference is not an onedoor-decision-ref/1 object, a step acted on a denied or proposed decision, a permit does not cover the action the step recorded (signature, action digest or lifetime), or a cited decision is not in the export you passed. Or the --decisions or --permit-issuers file can't be read.

**Fix:** Record a decision in every declared gated step (ctx.decision), cite a permitting decision for a step that acts, and record the action a permit covers; pass the decision export as --decisions and the issuer keys as --permit-issuers, in the form onetrace-verify --help names.

## unknown-command

`onetrace` was given a subcommand it doesn't have. Verifying a run is a separate command, `onetrace-verify`.

**Fix:** Run `onetrace --help` for the subcommands; to verify a run, `onetrace-verify <run folder>`.

## s3-extra

A run destination or a sink command names an `s3://` bucket, but the S3 client the sink needs is not installed. onetrace records no credential and no endpoint; the client reads those from its own environment and credential files.

**Fix:** Install the extra: `pip install 'onetrace[s3]'` (hash-locked in requirements-s3.txt), or give a local run destination instead of an `s3://` URL.

## s3-url

An `s3://` value is not a bucket and prefix: it must read `s3://bucket/prefix`, with a non-empty bucket and an optional prefix path.

**Fix:** Write the destination as `s3://bucket` or `s3://bucket/prefix`.

## sink

A finished run could not be uploaded to its `s3://` destination, or a sink command could not reach it: the bucket was unreachable, refused the write or the read, or an object's checksum did not match on the way back. The local run folder is kept.

**Fix:** Check the bucket and the client's own credentials, then `onetrace sink retry <run folder>`; the run is already recorded locally and loses nothing by a failed upload.

<!-- https://oneproof.dev/onetrace/verdicts.md -->

# The five verdicts

Every stage in a two-run comparison (`diff`, or `localize` given two runs) gets exactly one of
five verdicts. There is no sixth.

- **same** — the stage's output digest matches on both sides.
- **FIRST DIFFERENCE** — the first stage, in chain order, whose output digest does not match.
  There is at most one of these per comparison.
- **downstream** — comes after the first difference and is still different. The divergence
  propagated forward, which is the ordinary case: a stage that consumes a changed input usually
  produces a changed output.
- **reconverged** — comes after the first difference, but matches again. Different work
  happened to produce the same result. This is reported honestly rather than hidden, because it
  happens in real pipelines (a reranker that recovers the same top document from a differently
  ordered candidate set, say) and silently folding it into "same" would erase real information
  about what actually ran.
- **COULD NOT CHECK** — the stage can't be evaluated: a format version the verifier doesn't
  implement (a major other than 0, or a minor that isn't ASCII decimal digits), a declared boundary
  placed at or before it, a file the verifier can't read. This is not a pass,
  and it never gets silently treated as one.

## What a verdict is about

A verdict is always about a stage's **output** — specifically, `outputs_root` for a per-document
stage (see [the format page](format.md#two-roots-for-a-per-document-stage)), or the output
digest directly for a run-scoped one. A difference in the **instrument** that produced the
output (its name, version, or configuration digest), or in the **constants** a stage asserts,
never changes a verdict by itself. It shows up as an **annotation** alongside the ladder instead
— naming exactly what differs — so a `same` verdict never hides that *how* an output was
produced changed even when the bytes it produced didn't.

## The overall result, separately

Each verb also reports one word for the whole comparison, with its own exit code:

| Verb | Result words | Exit codes |
|---|---|---|
| `diff` | `identical`, `diverged`, `not comparable`, `refused`, `could not check` | 0, 1, 2, 3, 4 |
| `localize` (two runs) | same five words as `diff` | same five codes |
| `localize` (one run) | `clean`, `located` (plus `refused`/`could not check`) | 0, 1 (3, 4) |
| `reproduce` | each stage is `REPRODUCED`, `DIVERGED`, or `COULD NOT CHECK` | 0 all reproduced, 1 any diverged, 4 any could-not-check and none diverged, 3 refused |

`reproduce`'s own three outcomes are a different question from the five verdicts above: they
answer "if I re-run this stage's own code right now, does it produce what the receipt says it
produced," not "do two already-recorded runs match." `rederivable` is measured by actually
re-executing the stage, never declared; a stage that isn't rederivable keeps its originally
recorded outputs rather than being treated as broken.

## A run's verdict against a reference (onetrace 0.6)

A [reference](references.md) is a sealed record of a known-good run. `onetrace verdict RUN
--reference FILE` reads a finished run against one and gives each step one word, and the run its
own. A verdict is derived from the record and the reference's declared rule; nothing is inferred.

**The run's word** is one of four, with the same exit codes the other verbs use:

| word | means | exit |
|---|---|---|
| `identical` | every step matched | 0 |
| `diverged` | a step's output, setting or input differs from the reference; the verdict names the first such step | 1 |
| `refused` | the run is refused (it does not verify, or a step was held) | 3 |
| `could not check` | the verifier could not check the run, or the reference names a step the run does not record | 4 |

**Each step's word** is one of six, each with the record item it rests on:

- **same output** — the step's output digests match the reference.
- **output differs** — they do not.
- **setting differs** — the instrument's configuration digest, or a constant the rule counts,
  differs.
- **input differs** — a data input's digest differs.
- **slower than its range** — the step matched on output, setting and input but ran longer than the
  reference's timing range. Timing is a **view**: a step slower than its range is never a divergence
  and never fails a verdict.
- **could not check** — the reference names this step; the run does not record it.

Among the first three differences the order is output, then setting, then input: a step reads the
first that holds. The run `diverged`s at the first step that reads output, setting or input differs.

**onetrace-verify** gains the same verdict over a run and a reference, as rows beside its others:
`--reference FILE` adds one row per step — PASS (same output, or slower than its range), FAIL
(output, setting or input differs), or NOT-RUN (could not check) — and a row for the reference's
seal and, where it carries one, its signature. A nested run is checked against its own reference
with `--nested-reference FILE`, given its folder with `--nested-run`; with no reference for it, the
nested run reads NOT-RUN with the reason. A run verified without `--reference` reads exactly as
before. The verifier reads the reference with its own canonicalizer and imports nothing from the
SDK.

<!-- https://oneproof.dev/onetrace/references.md -->

# References: a sealed record of a known-good run

A **reference** is a sealed record of a run you vouch for, or of selected steps of it: per step,
the output digests, the settings, the data inputs and a timing range, with the rule for the step.
A later run is checked against it with [`onetrace verdict`](verdicts.md), which gives each step one
word and the run its own. A reference is its own file, format `onetrace-reference/1`; it is never a
receipt and never part of a run's chain.

## Making one

```
onetrace reference make RUN… --out FILE [--steps NODE …] [--mask FILE] [--sign FILE | --sign-env NAME]
```

- `RUN…` is one or more finished runs that verify. A run that is refused, or that the verifier
  could not check, is not a known-good run: making a reference from it is refused, naming it.
- The output digests, the settings and the data inputs are taken from the first run; the **timing
  range** is taken across every run given (a range needs more than one run — a single run gives a
  point, not a range, so a reference made from one run carries no timing).
- `--steps NODE` keeps only the named steps (by node id, `name` or `name#instance`); the default is
  every step of the first run.
- The reference is **sealed** as a receipt is: its `seal` is the SHA-256 content digest over its
  canonical bytes. `--sign FILE` or `--sign-env NAME` signs the seal with an Ed25519 key, the way a
  run's chain head is signed; a reader checks the signature as they check a run's.

```
onetrace reference show FILE
```

prints a reference as text: its seal, its signature, the runs it was made from, and per step the
outputs, inputs, settings and timing range.

## What a step holds

| field | what it is |
|---|---|
| `outputs` | the step's output digests, by name |
| `inputs` | the step's **data** inputs by digest (not the `receipt:` links to earlier steps, which are the run's structure) |
| `settings` | the instrument's configuration digest and the stage's own constants (onetrace's own `onetrace.*` constants are not settings) |
| `timing_ms` | the range `{min, max}` across the runs given, when more than one gave a readable time |
| `rule` | `exact` (the default), or a mask |

## The rule

The default rule is **exact**: a step's output digests, its settings and its data inputs must match
the reference. `--mask FILE` (an [`onetrace-mask/0.1`](verbs.md) file) names settings not to count
as a difference; the reference records the rule in full, so a reader sees exactly what was not
counted. Outputs are always compared by digest — a reference holds digests, which a value mask has
nothing to act on — so a tolerance on a numeric output waits for the per-step values a later
release records, not this one.

## A pipeline may hold several

A pipeline's golden set is its references — one per release, say. Each is a separate file; a run is
checked against whichever one you name.

## What a reference does not claim

- A step matching its reference shows it did what the known-good run did, not that either was right.
- A reference is a statement that this run is the one to compare against, not that it is correct.
- The seal shows the reference was not changed after it was made; the signature shows who made it,
  under the trust the reader places in the key. Neither shows the run it was made from was correct.

<!-- https://oneproof.dev/onetrace/variation.md -->

# Variation: repeat runs and the profile

Monitoring shows where runs depart; it does not make a step repeatable. These two things help a
team see where variation enters, and whether a fix holds.

## Repeat-run sets

Record several runs of the same input as a set, so they can be read together:

```python
@ot.run(repeat_of=RUN_ID)          # this run repeats RUN_ID; Recorder(repeat_of=…) does the same
def main():
    ...
```

or run a command N times and link the runs it writes:

```
onetrace repeat -n N --runs-root DIR -- CMD…
```

The first run is the set's anchor; runs 2…N are started with `ONETRACE_REPEAT_OF` set to the
anchor's id, so each records the manifest member `repeat_of`. A run that repeats another records
`repeat_of`; a run that repeats nothing records nothing new, and is byte-identical to before.

## A step that may vary

A step may declare that its output is expected to differ between runs:

```python
@ot.stage("plan", instrument=…, may_vary=True)
def plan(…):
    ...
```

The declaration is recorded on the receipt (`may_vary`) and shown by the profile. It is never used
to hide a difference: a `may_vary` step that differs is still counted and still named.

## The profile

```
onetrace profile RUN…
```

reads a set of two or more runs and shows, per step, how often its output differs across the set
and of what kind:

- the runs are grouped by their output digests; the largest group is the set's common output, and a
  run outside it is one whose output differs;
- each differing run is classified against the common group: its **setting differs**, its **input
  differs**, or neither — **output only**, which the step is then marked **"varies on its own"**
  for (the question "does it differ even when nothing changed?").

Counts only; no score or grade.

## Whether a fix holds

After a change meant to make a step repeatable (a pinned seed, temperature 0, a fixed prompt, a
cached tool result), re-run the set and read the two profiles side by side:

```
onetrace profile --before SET… --after SET…
```

It shows, per step, how often the output differed before and after (for example, `plan`: 4 of 5
before, 0 of 5 after), and **names the setting that changed between the sets**. It does so as a
comparison does: it names the setting; it does not say the change made the difference.

A step that now matches across its repeats can be given an exact-match rule in a
[reference](references.md); one that still varies keeps a mask, declared, never guessed.

## What a profile does not claim

- A profile describes the repeats recorded, not every future run.
- Matching in N repeats does not prove a step is deterministic.

<!-- https://oneproof.dev/onetrace/streaming.md -->

# Streaming the record as it is written

A run may send each receipt to a console as it is written, so the console can show the run filling
in step by step while it is still going. Streaming **shares the record as it grows; it is not the
record.**

## Turning it on

```python
@ot.run(stream_to="https://console.example/ingest")
def main():
    ...
```

or set `ONETRACE_STREAM_TO` in the environment, so a deployment can turn it on without changing the
code. `Recorder(stream_to=…)` does the same.

## What is sent

As each receipt is written, its bytes are sent to the endpoint as an HTTP POST. The body is the
receipt's bytes and nothing else; the run id and the receipt's position in the chain (its index and
digest) travel as headers (`X-Onetrace-Run-Id`, `X-Onetrace-Chain-Index`, `X-Onetrace-Chain-Digest`).

Streaming **changes no receipt and adds nothing to the chain.** A run streamed and a run not
streamed write the same receipts and the same chain head; the only difference is in the manifest,
which records the stream's outcome.

## When the endpoint cannot be reached

A slow or dead endpoint never holds the run up. The receipt is written, the chain goes on, and the
stream for that receipt is recorded as not delivered. When the run closes, the manifest says how
many receipts were delivered and how many were not:

```json
"stream": {"delivered": "3", "not_delivered": "2", "note": "the stream was not delivered"}
```

The endpoint URL is not recorded. A failed stream changes no receipt: the run records as before and
verifies as before.

## The other mode: watching the folder

A console on the same machine can instead watch the run folder and read each receipt as it appears;
there is nothing to configure in the SDK for that. Streaming is for a console that is not on the
machine the run is on.

## What streaming does not claim

- Streaming shares the record as it grows; it is not the record. Only the written run folder, read
  and verified, is the record.
- A receipt that reached a console is provisional until the run closes and its whole chain verifies.

<!-- https://oneproof.dev/onetrace/storage.md -->

# Storage: a run in a bucket

A run folder is immutable files whose integrity is in the files themselves — the chain, the
digests, the signature. So a finished run can live in an S3-compatible bucket, apart from any
console, and a reader verifies it on every read. The bucket **verifies nothing; the reader does.**

The client is the optional `onetrace[s3]` extra. Without it, an `s3://` destination is refused,
naming the extra; a local destination needs none of it.

## Sending a run to a bucket

Name the destination with `ONETRACE_RUNS`, so a deployment turns it on without changing the code:

```
ONETRACE_RUNS=s3://my-bucket/runs  python pipeline.py
```

or on the run itself:

```python
@ot.run(runs="s3://my-bucket/runs")
def main():
    ...
```

`Recorder(runs=…)` does the same. A local path there (or none) writes runs locally as always.

The run is **written and sealed locally first**, exactly as a local run is. When it closes, the
sealed folder is uploaded, **the manifest last**, so the last byte to reach the bucket is the one
that names the chain head: a reader sees a whole run or no run, never half of one. Each object's
checksum is set to the file's sha256 — the S3 checksum header where the endpoint stores one, else a
`<key>.sha256` sidecar beside the object.

The manifest gains a `sink` member saying whether the upload was delivered:

```json
"sink": {"delivered": "true", "objects": "7"}
```

The **bucket, the endpoint and the credentials are never recorded.** The client reads its
credentials from its own environment and credential files (the standard ones the S3 client uses),
never from onetrace's own configuration; onetrace records no credential and no endpoint in the run.

A run with no sink writes the same bytes it always did: the `sink` member is absent, and the record
format is unchanged.

## The index object

Beside the runs, one small object per run — `<prefix>/_index/<run id>.json` — carries the run id,
the pipeline, the chain head, the first step's instrument, the run's time, the verdict it recorded
(when it recorded one) and the kept-files count. A console lists runs from these without fetching a
whole folder.

## When the upload does not go through

A failed upload never loses the run: it is **recorded, not silent.** The failure is reported on
stderr and in the manifest's `sink` member, and the local folder stays:

```json
"sink": {"delivered": "false", "objects": "0", "note": "the run was not delivered",
         "reason": "EndpointConnectionError"}
```

Upload it later, from where it was recorded:

```
onetrace sink retry ./runs/<run id>
```

`sink retry` reads the destination from a `.sink` file beside the folder (a local operational file,
not part of the record and never uploaded), or takes `--to s3://bucket/prefix`. A delivered run's
local copy stays unless `--sink-remove` is given.

## Reading a run back

Verify a run in the bucket as you would a local one:

```
onetrace sink verify s3://my-bucket/runs/<run id>
```

It fetches the run into a temporary folder, checks each object against its stored checksum or its
`.sha256` sidecar, and runs the verifier over it, reporting as the verifier does. `onetrace-verify`
itself stays file-only: it reads a folder, never a bucket.

## Streaming into the same bucket

A run may stream its receipts to the same bucket as they are produced
(`stream_to="s3://my-bucket/runs"`, or `ONETRACE_STREAM_TO`), so a run interrupted partway still
leaves its partial chain in storage. A streamed receipt is written under the run's own prefix,
**byte-identical to the one the sealed upload carries.** See [streaming](streaming.md).

## Lifecycle lives in the bucket, not in onetrace

The runs are plain objects, so the bucket's own lifecycle rules — not onetrace — move old runs to
colder storage, or delete them, when a team decides. A team can keep the small receipts for years
and let the larger kept inputs and outputs expire sooner, by prefix; the verifier reads a run whose
kept files are gone as "outputs not kept", a verdict it already knows. onetrace writes the runs and
reads them; it does not manage the bucket.

## What the sink does not claim

- **A bucket verifies nothing; the reader verifies on every read.** Storage location does not change
  what a record proves. A tampered object is caught when a reader verifies the run, not at rest.
- **A delivered run is the same record as the local one.** The sink uploads the sealed folder and
  changes no receipt; the only addition is the manifest's optional `sink` member, which names no
  endpoint and no credential.
- **An object's checksum guards the transfer, not the record.** The chain, the digests and the
  signature are what prove the run; the per-object sha256 only catches an object damaged in the
  bucket, on the way back.

<!-- https://oneproof.dev/onetrace/decisions.md -->

# Recording which onedoor decision allowed an action

A step that takes an action can record which [onedoor](https://onedoor.example) decision allowed
it, and the record proves it: anyone can check, with no network calls, that the decision exists
unaltered, that it permitted this step, and, where a bound permit was used, that the permit covered
exactly the action the step recorded. onetrace never imports onedoor — the small, stable pieces it
needs are vendored as standard-library code (`onetrace.onedoor_ref`) and tested against onedoor's
own published vectors.

## Declaring the gated steps

A run declares which of its steps must be gated:

```python
@ot.run(stages=["approve_payment"], gated_stages=["approve_payment"])
def main():
    ...
```

`Recorder(..., gated_stages=[...])` does the same. Each gated stage must be a declared stage. The
declaration is written into the run's manifest. A gated step that records no decision is a finding
the verifier names, not a silence. A step not declared gated may still cite a decision.

## Recording the decision inside the step

Inside a decorated step, after onedoor answers and before the action:

```python
ot.decision(ref, permit=None, action=None)
```

- `ref`: the `onedoor-decision-ref/1` object onedoor returns (`request_id`, `decision_digest`,
  `verdict`, `issuer`), recorded verbatim. A step may record more than one — a proposal, then its
  resumption — and they are kept in call order.
- `permit`: optional, the bound permit's compact JWS, where one was issued. It is recorded verbatim
  so its signature can be re-checked later.
- `action`: optional, the action object the step is about to send. It is recorded **by digest
  only** — the digest the bound-permit profile uses (domain tag `aadp:action:v1`, one zero byte,
  the JCS form of the object, SHA-256, base64) — never the object itself. An action object's numbers
  are **decimal strings**, as onedoor's vectors have them (`"amount_eur": "40.00"`); a float in an
  action is refused and named, never canonicalised.

These go in the receipt's optional `authorization` block. A run that records no decision writes
receipts byte-identical to 0.4.0.

## What onetrace-verify checks

Five rows, each PASS, FAIL or NOT-RUN, never guessed. They appear only when a run declares gated
stages or cites a decision; a run verified without any is read exactly as before.

| row | PASS | FAIL | NOT-RUN |
|---|---|---|---|
| gated steps cite a decision | every declared gated step cited one | a gated step cited none | no `gated_stages` declared |
| references are well formed | each is `onedoor-decision-ref/1` | one is malformed | no references |
| a step that acted cites a permitting decision | its last reference's verdict is `permit` | it acted on a `deny` or `propose` | no references on an acting step |
| the permit covers the recorded action | the signature is valid for an issuer in `--permit-issuers`, the action digest equals the step's recorded action, and the step ran within the permit's lifetime | any of these fails | no permit recorded, or no issuer key given |
| the decision is in the export | `--decisions <export.jsonl>` (with its `.sha256` sidecar) given, and every reference matches a row by digest | a digest mismatch, not in the export, or the export is damaged | no export given |

The last row reuses onedoor's own five outcomes. With no export and no permit, the record can still
show that a gated step cited a permitting decision; the rows that need more read NOT-RUN, so nothing
reads as checked that was not.

```
onetrace-verify --decisions decisions.jsonl --permit-issuers issuers.json runs/<run_id>
```

The permit issuer's Ed25519 public key is read from `--permit-issuers`, an `onedoor-issuer-trust/1`
file, separate from the run-signing `--trust` file: the permit issuer is a different trust domain
from the run's signer, and the `onetrace-trust/0.1` file carries key fingerprints, not public keys.
The run-signing `--trust` file is never read for an issuer, and an issuer key is never read from the
run. The file names each issuer and its 64-hex Ed25519 public key:

```json
{
  "format": "onedoor-issuer-trust/1",
  "issuers": [
    {"issuer": "https://onedoor.example/payments-issuer",
     "alg": "ed25519",
     "public_key": "03a107bff3ce10be1d70dd18e74bc09967e4d6309ba50d5f1ddc866412553..."}
  ]
}
```

When `--decisions` is given without `--permit-issuers`, the permit row reads NOT-RUN, naming the
missing `--permit-issuers` file, so a permit is never read as checked when its issuer key is absent.

## What this does not claim

- A citation shows that the run names an existing, unaltered decision. It does not show that every
  action was gated, only that every declared gated step cited one.
- A permit check shows the decision covered the action object the step recorded. It does not show
  that the receiving system received exactly that object; that is the recipient's own check.
- Nothing here says a decision was right.

<!-- https://oneproof.dev/onetrace/span-context.md -->

# The span context

When a step runs inside an active OpenTelemetry span, its receipt records that span's W3C trace id
and span id, and the run's manifest records the run's own `traceparent`. This is the one thing the
record keeps about a trace: where the run and its steps ran, so a later reader can line the record
up with the spans an app emitted — nothing more.

## What is read, and from where

- Where an OpenTelemetry package is installed and a span is recording, its active span is read.
- Where none is, the `TRACEPARENT` environment variable is read instead.
- With no active span and no `TRACEPARENT`, nothing is recorded and the receipt is byte-identical
  to 0.4.0.

No OpenTelemetry package is required: onetrace reads the active context where it can and the environment
otherwise. No network call is made.

## What is recorded

- Each step's receipt gets an optional `trace` block: `{"trace_id": "<32 hex>", "span_id":
  "<16 hex>"}`, the span active when the step ran.
- The run's manifest gets an optional `traceparent`: the full W3C traceparent the run ran in
  (`00-<32 hex trace id>-<16 hex span id>-<2 hex flags>`).

A `TRACEPARENT` that is present but malformed — not a W3C traceparent, or with an all-zero id — is
refused and named, never recorded: a broken trace is not written into the record.

## What this does not claim

- **A trace is a view, not evidence.** It may be sampled, dropped, rewritten or re-timed by a
  collector. The span context says which trace the run and its steps ran in; only the record proves
  the step's inputs and outputs.

<!-- https://oneproof.dev/onetrace/nesting.md -->

# Nesting

A record shows which step called which, for stages and for whole runs, so that an agent that calls
a sub-agent (or a LangGraph subgraph) is one record a reviewer can walk down. A run with no nesting
writes receipts byte-identical to 0.4.0.

## Stage inside stage

A stage called while another is running records `parent` (the caller's node id) in its receipt, and
the caller's receipt lists its `children` in call order. A child's return given to another stage as
an argument is linked the usual way, as that stage's input, by digest. A child's return the caller
uses in its own body is not recorded as an input of the caller: the caller's receipt names the
child in `children`, and nothing more. One context variable holds the step running now; the span
context and nesting both read it.

## Run inside run

A decorated run started inside another's stage keeps its own folder and chain. Its manifest records
`parent_run` (the outer run's id) and `parent_stage` (the calling stage). The outer run records the
inner run as an input of the calling stage — named `run:<inner run id>`, with the inner run's chain
head as its digest — so the outer run is no longer empty.

A LangGraph subgraph node is an ordinary decorated stage, so it records as a nested step when it
runs inside another; its state-key edges stay as they are.

## The depth limit

Nesting is recorded up to a fixed depth (`onetrace.emit.NEST_DEPTH_LIMIT`, **32**; 1 at the top
level). A call nested deeper than the limit is refused and named, never left out, so a runaway
recursion cannot fill a record folder.

## Comparing nested runs

`onetrace diff` and `onetrace localize` name the first difference in chain order. A step's receipt
is written when the step ends, so a stage called inside another stage comes before its caller in the
chain. What the comparison prints depends on the shape:

- **A run started inside a stage.** The calling stage records the inner run as its input
  `run:<inner run id>`, so the calling stage is the first difference. Given both inner runs' folders
  (`--nested-run PATH`, once for each), the comparison descends into the inner run and names its first
  differing step: `inside <step>, into run <id> at <child>`. Without them it says `nested run <id>:
  folder not given`.
- **A stage called inside another stage.** Each stage is compared in its own right. When the child's
  output differs, the comparison reports two first differences, the child first in chain order and
  then its caller, and prints no `inside` line. A child called in one run only is reported as present
  in that run only, and its caller, whose output is the same, as `same`.

The `inside <step>, at <child>` line is printed only when the first difference is a step whose
recorded children differ from the other run's. Neither stage-inside-stage shape above prints it.

### A stage inside a stage, compared

`plan` calls `price_lookup` inside its own stage. Record it twice, with two prices:

<!-- the stage-in-stage example -->
```python
# order.py -- record it twice: python order.py 10, then python order.py 12
import sys

import onetrace as ot
from onetrace.emit import Instrument

PRICE = int(sys.argv[1])
TOOL = Instrument("order", "tool", "1.0.0", {})


@ot.stage("price_lookup", instrument=TOOL)
def price_lookup(item):
    return {"item": item, "price": PRICE}


@ot.stage("plan", instrument=TOOL)
def plan(item):
    p = price_lookup(item)            # a stage called inside plan's stage
    return {"item": item, "total": p["price"] * 2}


@ot.run(stages=["plan", "price_lookup"])
def main(item):
    return plan(item)


main("widget")
```

```
ONETRACE_RUN_ID=order-a python order.py 10
ONETRACE_RUN_ID=order-b python order.py 12
onetrace localize runs/order-a runs/order-b
```

The comparison, as `localize` prints it (exit 1, diverged):

<!-- localize output -->
```text
localize: diverged

node            A     B     verdict
--------------------------------------------------------------------
intake          1     1     same
price_lookup    2     2     FIRST DIFFERENCE
                            output return.json differs
plan            3     3     FIRST DIFFERENCE
                            output return.json differs

first difference  price_lookup, plan   (2 branches)
comparisons performed 3 of 3 nodes; same 1, FIRST DIFFERENCE 2, downstream 0, reconverged 0, COULD NOT CHECK 0
```

`onetrace diff runs/order-a runs/order-b` prints the same table and the same `first difference`
line under `diff: diverged`, and also exits 1.

## Verifying nested runs

onetrace-verify adds one row, **"nested runs match their recorded heads"**: give each inner run's
folder with `--nested-run PATH`, and the inner run's own chain head must equal the head the outer
run recorded for it.

- PASS: every recorded inner run whose folder was given matches its recorded head.
- FAIL: a recorded head differs from the inner run's actual chain head, or an inner run's folder was
  given but its manifest cannot be read (a damaged inner chain).
- NOT-RUN: no nested run was recorded, or no folder was given.

```
onetrace-verify --nested-run runs/<inner_run_id> runs/<outer_run_id>
```

## What this does not claim

- **Nesting is recorded for calls through decorated stages only.** A call that bypasses the
  decorator is not in the record, as today. `parent`, `children`, `parent_run` and `parent_stage`
  name the calls that went through `@ot.stage` and `@ot.run`, and nothing else.
- **A comparison does not descend into a stage called inside another stage.** It reports that stage
  as a first difference of its own, as shown above; it descends into a run started inside a stage
  only when both inner runs' folders are given.

<!-- https://oneproof.dev/onetrace/model-call-record.md -->

# The model-call record

A **model-call record** is one canonical JSON document per model call, written as an ordinary output
of the stage that made the call. It is evidence beside the stage receipt: it never changes the
receipt format and never adds a member to the receipt or the chain. It never changes the reading of a
run's receipts; its own verifier rows read as every other row does, and a FAIL on one fails the run
like any other FAIL. A run that writes no record is byte-for-byte the run it would be without the
feature.

It is written by `onetrace.model_call`, installed with `onetrace[model-call]`. The engine adapters
import vLLM or transformers only when used, so the engine stays your own installed version; onetrace
pins no engine. The transformers adapter is tested only where torch and transformers are installed
(`tests/test_model_call_capture.py`, a tiny model built from a config on the CPU); its tests skip
without them. CI runs them in a job of their own on a CPU-only torch, pinned by hash. The record's schema is `docs/model_call_record.schema.json`, labelled
`model-call-record/1.0`.

## What a record holds, and how honestly

Every scalar in a record is a string. Every field that describes the call is a **value object** with a
`status`, a `source`, and — where there is one — a `value`. A field the writer cannot fill is written
as `not observed` or `not applicable`, with its reason in `source`: never left out, and never given a
default.

The five states:

- **`observed`** — read at call time; `source` says from where.
- **`declared, not observed`** — given by the caller and not read back from the engine; `source` says
  what it is.
- **`declared, contradicted`** — declared, and a different value was read; both are kept (`value` the
  declaration, `observed_value` the reading).
- **`not observed`** — the writer cannot see it; no value, never a default; `source` says so.
- **`not applicable`** — the field has no meaning for this engine or call; no value; `source` says
  why.

Four fields are written by the helper itself, as plain strings with no state — they describe how the
record was written, not the call: `format`, `call.kind`, `call.api`, and `credential`.
`credential` is `present` or `absent` — never the key, its digest, or any part of it.

### The record says only what it observed

The record is written so that a status is never a claim the writer did not make:

- **Settings the engine did not expose are declared, not observed.** A self-hosted call's
  `sampling.*` is the caller's own sampling parameters as built, not read back from the engine, so it
  is `declared, not observed`. A value a caller hands the adapter for a `runtime.*` field is
  `declared, not observed` for that field — never shown as observed.
- **The runtime is read in the calling process.** `runtime.torch`, `runtime.cuda`,
  `runtime.gpu_name` and `runtime.compute_capability` are read from `torch` in the calling process
  and are `observed` from it; a field not read there is `not observed`. The driver is not read from
  `torch`: a driver the caller gives the adapter is `declared, not observed`, as a caller's value for
  any `runtime.*` field is; without one, the driver is `not observed`; it is never `observed`. On a CPU run, the transformers adapter writes the GPU runtime fields
  (`runtime.cuda`, `runtime.gpu_name`, `runtime.compute_capability`, `runtime.driver`) as **`not
  applicable`** — `not applicable` is one of the record's five states, and it is the truer word than
  `not observed` for a field that has no meaning on a CPU run.
- **A model digest names the directory it hashed, not the engine.** `model.files_digest` and
  `tokenizer.digest` are the digest of the directory given to the adapter; the `source` says "the
  directory given to the adapter; not read from the engine." Nothing ties the hashed directory to the
  weights the engine actually loaded, and the record does not claim it does.
- **A constant says it is a constant.** `engine.name` is `observed` with the source "a constant
  written by this adapter."
- **The filler count is the caller's.** `batch.fillers` is `declared, not observed`: how many of the
  submitted requests the caller said were fillers.
- **A hosted call invents nothing.** Every internal field of a hosted call (`engine.*`, `runtime.*`,
  `batch.*`, the token-id digests) is `not observed`: the provider does not expose it. When the
  request names no model, `model.requested` is `not observed`, "the request names no model" — not an
  empty declared string. On an HTTP error, `output.finish` is `observed`, `error (HTTP <status>)`,
  with the source "the response's HTTP status": the status is what was observed, not a finish reason
  the provider did not send.

### The fingerprints

`fingerprints` is a list of distribution digests. A transformers prefill records an `exact` form (the
raw bytes of the last position's logits) and a `tolerant` form (the top-16 token ids of that
position, rounded). A vLLM call records a `topk_exact` and a `tolerant` form over the first generated
token's top-16 log-probabilities. A hosted call records a `tolerant` form over the provider's
first-token log-probabilities, when the response carries at least 16.

## Several calls in one stage

The first record a stage writes is `model_call.json`, then `model_call.2.json`, `model_call.3.json`,
and so on, in write order. The package refuses a name it has already used in a stage before it writes,
so a repeat never reaches onetrace.

## The capture levels, and the steps file

A call's **answer** — its chosen token ids — is the model's own output, not the operator's. The
record never holds a token id or any text, at any capture level.

When a per-step capture (L1: the chosen token ids; L2: the top-k candidate ids) is taken, it is
written as a **separate output**, `model_call.<n>.steps.json`, classed **`model-generated`**, and the
record names it only by **digest**, in the optional `steps` field. The token ids live in that
model-generated file; the operator-authored record holds a digest of it and nothing more.

A prefix-digest chain over the token ids is not a way to hide them: each step's digest, with the
model's vocabulary, **reveals the tokens to anyone with the vocabulary**. The steps file is treated as
the model's output, classed `model-generated`, for that reason.

The cost of taking L1 or L2 is **not measured**. L3 (a full per-step capture on the serving engine) is
**not measured**.

## Comparing two records

`onetrace.model_call.compare(a, b)` reads two records and returns one of three results, and nothing
else:

- **`same`** — the compared fields are equal under the stated mode.
- **`diverged`** — naming the first field that differs.
- **`could not compare`** — naming each condition that was not met, in order.

A comparison names the mode it applied (1, 2 or 3) and what it rested on. It measures nothing. vLLM
with batch-invariant mode not observed on, at a submitted batch size over 1, **is not compared**: the
reason says that, and claims no property of such a configuration. The evidence for that condition is
one measurement, named in full, applied to every such record with no other configuration having been
measured.

When the two sides' per-step token-id sequences are given to `compare` and the result is `diverged`,
the first differing step is added as a **line of that diverged result** — never as a fourth result.
The three results stand.

Comparing two records is `onetrace.model_call.compare`: the verbs `onetrace diff` and `onetrace
localize` do not read records in 0.6.0 or 0.6.1.

## The verifier's rows

A verifier that reads a run gains rows **only for a run that holds a record**; a run with no record
gains no row, and its output is unchanged. For each record the run holds, the rows read exactly as the
receipt rows do — `PASS`, `FAIL` or `NOT-RUN`, never a guess, the matched text of a credential never
echoed:

- the record parses and its bytes are canonical;
- the record format is one the verifier implements (`model-call-record/1.0`); a format it does not
  implement reads `NOT-RUN`, naming it, and the run's exit is unchanged;
- the fields are complete and well formed;
- no field holds a value where it is `not observed` or `not applicable`;
- no credential material: an authorization-header shape in any string is a `FAIL`, by structure only —
  the verifier is never given the key.

## The model link

`ot.model_from(provisioning_run, stages=[...], serve_stage=...)`, passed as `@ot.run(model=...)`,
links a run to the run that fetched, optionally quantised and served its model — the same shape as the
corpus link, one hop further. The link is the provisioning run's chain head, recorded on exactly the
named steps; the provisioning run is verified when the query run starts, and refused if it does not
verify, as a corpus link is. Following a model link one hop in a comparison, and cross-checking a
record's `model.files_digest` against the provisioning run's own digest, are later work and are not
done yet.

## What this does not claim

- **onetrace does not read the model's reasoning.** The record holds no hidden reasoning.
- **A record is not a correctness claim.** It does not check whether an answer is semantically
  correct.
- **A differing value is not a cause.** The record names no causality from a differing value alone.
- **A record does not reproduce across machines.** It claims no reproducibility across GPUs, versions
  or engines.
- **A record says nothing about a hosted provider's internals.** Those are `not observed`.
- **Record equivalence is not correctness**, and `same` covers only the compared fields.
- **The comparator's not-repeatable condition rests on one measurement** and fits no tolerance; it
  claims no cause.
- **A digest is not a commitment.** A digest in the record names a value; it does not bind the writer
  to anything, and is not a proof of what was served.
- **The companion result was measured at one arrangement.** That a filler request changed a target's
  answer was measured at batch 4 only, with the target first, companions no longer than the target,
  and no fillers beyond the companions; it is not a claim about other batch sizes or arrangements.
- **The tolerant form is not a detector of small numeric changes.** It is built to miss them; a small
  numeric change can leave the tolerant form unchanged.
- **The exact form is the full row for a transformers prefill, and the top 16 unrounded for a vLLM
  call.** It is not the full vLLM distribution.
- **The per-step capture holds the model's token ids.** It is `model-generated`, kept in a separate
  output, and the record names only its digest; a prefix-digest chain over token ids reveals the
  tokens to anyone with the vocabulary. The cost of L1 or L2 is not measured, and L3 is not measured.
- **The verifier does not check that a record's `steps` digest names the steps output.** The steps
  file is an ordinary model-generated output the chain covers; that the record's `steps` digest is
  that output's digest is not checked by a verifier row.

## The schema

`docs/model_call_record.schema.json`, labelled `model-call-record/1.0`, is the one schema. It uses
only the validation keywords a strict reader applies, so the reader can read it; it declares the
optional `steps` capture member; `x-groups` groups the fields for a reader, with a group for the
capture member; and `x-most-bytes` bounds a record's size, with its reason (the record holds digests
and short value objects, and the per-step capture lives in a separate output, so the record does not
grow with capture depth).

The wheel carries a byte-identical copy as package data,
`onetrace/model_call/model_call_record.schema.json`, readable with
`importlib.resources.files("onetrace.model_call").joinpath("model_call_record.schema.json")`.

<!-- https://oneproof.dev/onetrace/non-claims.md -->

# What onetrace does not claim

This list exists because every one of these is an easy thing to assume onetrace does, and it
doesn't.

- **onetrace does not check whether a pipeline's output is true.** It checks what was recorded,
  and whether an independent verifier can confirm that record is internally consistent and its
  chain unbroken. A pipeline that faithfully records a wrong answer produces a record that
  verifies cleanly — verification is not a truth oracle.

- **A trust class is not a correctness claim.** `operator-authored`, `model-generated`, and
  `externally-sourced` describe *provenance* — where a byte came from — not whether it's right.
  `model-generated` doesn't mean "less trustworthy"; `operator-authored` doesn't mean "correct."

- **`same` does not mean "nothing changed."** It means a stage's output digest matched. The
  instrument that produced it, its configuration, or an asserted constant can all have changed
  without moving the verdict — that's reported as an annotation, deliberately kept separate from
  the verdict itself (see [the verdicts page](verdicts.md)).

- **`reconverged` is not a bug in the comparison.** Two different computations landing on the
  same output is a real thing that happens; onetrace reports it as its own verdict rather than
  either hiding it inside `same` or forcing it to read as a difference.

- **An unanchored run's `originality` check reporting `NOT-RUN` is not a failure.** It means
  exactly what it says: this verifier can establish that a record is internally consistent, not
  that it was first published by whoever holds it. Anchoring (`onetrace anchor`: a timestamp
  proof over the chain head, from a party outside the recorder) is a separate record beside the
  chain; without it, originality is a question onetrace is honest about not answering, not one it
  answers `PASS` on a technicality.

- **An anchor does not show when the run happened, that its output was correct, or who wrote
  the record.** An anchor is a timestamp proof over a run's chain head, from a party outside the
  recorder. It establishes that this exact record existed no later than time T, as attested by
  the anchor's source, under that source's trust assumptions. It does not show:
  - when the run happened (only that the record existed by T);
  - that the output was correct;
  - who wrote the record (that is a signature's job, below).

  An RFC 3161 anchor is only as good as the time-stamp authority's certificate and root you
  pin: the verifier checks a token against those, never against a certificate the token carries.
  An OpenTimestamps anchor checked against block headers you supply is only as good as the source
  of those headers: the verifier trusts the heights in your header file, and checks each header
  only for its own proof-of-work target, not that it is on the chain everyone else follows. The verifier reads each anchor file once and checks and reports from
  those bytes, but a folder someone else is writing to can hold something else the next moment:
  verify a copy you control, not a folder someone else is writing to.

  When `onetrace anchor` can't store a record, its reason can be wrong in one case. On a system
  without `/proc/self/fd` (macOS, for example), if someone moves the run folder away and puts an
  empty folder, or a link to somewhere else, at its old name just as the record is being linked,
  the reason says the temporary file's name was moved, though the file is intact in the moved
  folder. What it says about the outcome stays true: nothing is stored. Anchor a run folder nobody
  else is writing to.

- **A signature does not show that the key wasn't stolen or misused, that the run's output is
  correct, when the record was signed, or who the person behind the key is.** A signature is an
  Ed25519 signature over a run's chain head, by a recorder's key: it shows that the holder of that
  private key signed this exact record. A signature does not show:
  - that the key wasn't stolen or misused;
  - that the run's output is correct;
  - when the record was signed (that is an anchor's job);
  - who the person behind the key is. The verifier knows keys, not people; your trust list is how
    you connect a key to a name, and a valid signature by a key not on it is never a pass.

  The `signer` label in a signature record is the signer's own claim, and is only ever reported
  as claimed.

- **`reproduce` does not mean every stage is infinitely re-runnable.** A stage that calls a model
  with no fixed seed, or reads from a source that's since changed, is legitimately not
  rederivable — that's `COULD NOT CHECK`, with the reason named, and the stage's originally
  recorded outputs are kept rather than discarded.

- **onetrace does not protect against a write-access attacker who can also recompute every
  downstream digest and chain link.** Rewriting history undetected requires forging or replaying
  bytes past the verifier's own checks; closing the remaining gap (an attacker with full write
  access to storage, patient enough to recompute everything downstream) is what anchoring is
  for (manual, section 15): a rewrite changes the chain head, and an anchor made before it no longer
  binds to the record. An anchor is not part of the chain, though: the same attacker can delete
  `anchors/`, and the run still passes. Anchoring closes the gap only for a reader who expects an
  anchor, or holds one independently of the run's own storage.

- **The format is not finished.** Format major version `0` (the current `stage-receipt/0.2`) is
  specified by an Internet-Draft that hasn't reached RFC status. Required members, canonical-form
  rules, and the rejection vectors may still change before major version `1`. Format major
  version is carried in every record and checked explicitly — a verifier that doesn't implement a
  record's version (a major other than 0, or a minor that isn't ASCII decimal digits) says so, by
  name, rather than guessing.

- **`reproduce` does not claim bit-exact floating-point reproduction on every machine.** A stage
  whose own code depends on numpy/BLAS (an embedding step, say) is claimed rederivable only on
  the environment recorded in the run — a different machine's BLAS build can legitimately produce
  different bytes for the identical code and inputs, which `reproduce` reports honestly rather
  than asserting past.

- **onetrace does not claim identical test behavior across every Python patch version on every
  OS.** CPython 3.10.11's own random-number-generator initialization on Windows is known to fail
  outright on some current runner images, unrelated to onetrace — which fixtures reproduce there
  is a property of that Python build, not of the format or the SDK.

- **The ingest-r1 PDF stage requires poppler's `pdftotext`, and refuses any other build by
  design.** `pdftotext` from a different vendor (xpdf, say) is not silently accepted as
  equivalent, even when it runs and produces text — ring 4's own emission checks the reported
  converter identity before writing a single receipt and refuses to run at all if it isn't
  poppler's. This is not a portability gap to be worked around; the converter's identity is part
  of the recorded configuration, and a different vendor is a different, unconfirmed instrument.

- **The text diff does not say why recorded outputs changed, or that either version was
  correct.** `diff --text` shows what the two runs' recorded outputs contained, read only after
  each run verified and each file matched its receipt.

- **`diff --text` never shows an input's content.** Inputs are fingerprinted, not stored, so a
  changed input is named, with its trust class and digests, and nothing more.

## onetrace 0.5: onedoor decisions, span context, nesting

- **A citation shows that the run names an existing, unaltered decision.** It does not show that
  every action was gated, only that every declared gated step cited one.

- **A permit check shows the decision covered the action object the step recorded.** It does not
  show that the receiving system received exactly that object; that is the recipient's own check.

- **Nothing here says a decision was right.** onetrace records which onedoor decision allowed an
  action and lets anyone re-check it offline; it does not judge the policy, the decision or the
  action.

- **A trace is a view, not evidence.** It may be sampled, dropped, rewritten or re-timed by a
  collector. The span context a receipt records (a W3C trace id and span id) and the run's
  `traceparent` say which trace the run and its steps ran in; only the record proves the step's
  inputs and outputs.

- **Nesting is recorded for calls through decorated stages only.** A call that bypasses the
  decorator is not in the record, as today. `parent`, `children`, `parent_run` and `parent_stage`
  name the calls that went through `@ot.stage` and `@ot.run`, and nothing else.

- **A 0.5 receipt is not a new format.** onetrace 0.5 stays in format 0.3: a 0.4.0 verifier reads a
  0.5 receipt unchanged, ignoring the `authorization`, span-context and nesting blocks it does not
  know, and a run that records none of them is byte-identical to 0.4.0.

## onetrace 0.6: references and verdicts, variation, streaming

- **A verdict is derived from the record and a declared rule.** The rules are declared per step —
  exact digest by default, a mask where one is given — never inferred. A verdict names where a run
  parts from its reference; it does not say why, or that either run was right.

- **A step matching its reference shows it did what the known-good run did, not that either was
  right.** A reference is a statement that a run is the one to compare against, not that it is
  correct.

- **Timing is a view, never part of a verdict.** A step slower than its range is reported, but it is
  never a divergence and never fails a verdict.

- **A profile describes the repeats recorded, not every future run.** Matching in N repeats does not
  prove a step is deterministic. A step declared `may_vary` is shown as declared; the declaration is
  never used to hide a difference.

- **A before/after profile names the setting that changed between the sets; it does not say the
  change made the difference.** It is a comparison, as the others are.

- **Streaming shares the record as it grows; it is not the record.** Only the written run folder,
  read and verified, is the record; a receipt that reached a console is provisional until the run
  closes and its whole chain verifies. A failed stream changes no receipt.

- **A 0.6 record is not a new format.** onetrace 0.6 stays in format 0.3: a 0.5.0 verifier reads a
  0.6 run unchanged, ignoring the `repeat_of` and `stream` members in the manifest and the
  `may_vary` member on a receipt, and a run that records none of them is byte-identical to 0.5.0. A
  reference is its own file (`onetrace-reference/1`), sealed and signed as a run is, never a
  receipt.

## onetrace 0.6: the run sink

- **A bucket verifies nothing; the reader verifies on every read.** Storage location does not change
  what a record proves. A tampered object is caught when a reader verifies the run, not at rest.

- **A delivered run is the same record as the local one.** The sink uploads the sealed folder and
  changes no receipt; the only addition is the manifest's optional `sink` member, which names no
  endpoint and no credential.

- **An object's checksum guards the transfer, not the record.** The chain, the digests and the
  signature are what prove the run; the per-object sha256 only catches an object damaged in the
  bucket, on the way back.

## onetrace 0.6: the model-call record

The [model-call record](model-call-record.md) is evidence beside a stage receipt; it never changes the
reading of a run's receipts — its own verifier rows read as every other row does, and a FAIL on one
fails the run like any other FAIL. What it does not claim:

- **A record is not a correctness claim.** It holds no hidden reasoning, and does not check whether an
  answer is semantically correct. A differing value names no cause, and `same` covers only the
  compared fields. Record equivalence is not correctness.

- **A record does not reproduce across machines.** It claims no reproducibility across GPUs, versions
  or engines, and says nothing about a hosted provider's internals, which are `not observed`.

- **A digest is not a commitment.** A digest in the record names a value; it does not bind the writer
  to anything, and is not a proof of what was served. In particular, `model.files_digest` is the
  digest of the directory given to the adapter, not read from the engine: it does not show that the
  served weights are the hashed ones.

- **The comparator's not-repeatable condition rests on one measurement** and fits no tolerance. vLLM
  with batch-invariant mode not observed on, at a submitted batch size over 1, is not compared; the
  reason claims no property of such a configuration.

- **The tolerant fingerprint is not a detector of small numeric changes.** It is built to miss them; a
  small numeric change can leave it unchanged. The exact form is the full row for a transformers
  prefill, and the top 16 unrounded for a vLLM call — not the full vLLM distribution.

- **The companion result was measured at one arrangement.** That a filler request changed a target's
  answer was measured at batch 4 only, with the target first, companions no longer than the target,
  and no fillers beyond the companions; it is not a claim about other batch sizes or arrangements.

- **The per-step capture holds the model's token ids.** It is `model-generated`, kept in a separate
  output (`model_call.<n>.steps.json`), and the record names only its digest — the record itself
  holds no token id and no text at any level. A prefix-digest chain over token ids reveals the tokens
  to anyone with the vocabulary; it hides nothing. The cost of taking L1 or L2 is not measured, and L3
  is not measured.

- **The verifier does not check that a record's `steps` digest names the steps output.** The steps
  file is an ordinary model-generated output the chain covers; a verifier does not add a row checking
  that the record's `steps` digest is that output's digest.

<!-- https://oneproof.dev/onetrace/explain-format.md -->

# onetrace-explain: what it reads, sends, checks and writes

`onetrace-explain` (a separate package, `pip install onetrace-explain`) suggests where to look in a
comparison. **An explanation is a suggestion about a record, never a reading of it.** onetrace's
verdicts, exit codes and `onetrace summary` are what they are without it; the deterministic tools
never call a model and never read an explanation back. This page fixes the formats a reader of an
explanation (the console, a script) can rely on. Version 1, `0.1`.

## The command

```
onetrace-explain REPORT_DIR [--model PROVIDER:NAME] [--dry-run] [--include-text] [--redact NAME]... [--yes] [--out DIR]
```

`REPORT_DIR` is a folder `onetrace diff` or `onetrace localize` wrote with `--out`. Exit codes are its
own and say nothing about the runs: 0 an explanation or a dry run was written; 2 a usage error, or a
comparison that can't be explained (no report, a refused or could-not-check verdict, a run whose
chain head is not the one the report checked); 3 the model couldn't be reached, or replied with no
usable JSON. Never use it as a CI gate.

## The evidence packet, `onetrace-explain-evidence/0.1`

One JSON object, its bytes sorted-key, no spaces, UTF-8, built the same from the same report:

```json
{"format": "onetrace-explain-evidence/0.1",
 "report": {"name": "localize.json", "sha256": "sha256:…"},
 "runs": {"baseline": {"run_id": "…", "chain_head": "sha256:…", "format": "stage-receipt-chain/0.3"},
          "candidate": {…}},
 "items": [{"id": "node/load/constant/chunk_size", "kind": "constant", "name": "chunk_size",
            "baseline": "512", "candidate": "1024"}, …],
 "kinds": ["constant", "for", "input", …]}
```

Each item has an `id` (what a claim cites) and a `kind`:

| kind | id | holds |
|---|---|---|
| `verdict` | `comparison/verdict` | verb, envelope, exit, first difference |
| `for` | `comparison/for` | the `--for` node, its result, origins, not reaching, the path's verdicts |
| `end_outputs` | `comparison/end_outputs` | the report's end outputs |
| `inputs_differ` | `comparison/inputs_differ` | the inputs that differ, and `on_different_inputs` |
| `mask` | `comparison/mask` | the mask's sha256, how many settings it masked |
| `node` | `node/<node>` | the node's verdict and annotations |
| `instrument` | `node/<node>/instrument` | id, kind, version, config digest, in each run |
| `outcome` | `node/<node>/outcome` | the outcome class in each run |
| `upstream` | `node/<node>/upstream` | the nodes it read |
| `constant` | `node/<node>/constant/<name>` | a constant that differs, both values (`redacted: true` and both digests with `--redact`) |
| `input`, `output` | `node/<node>/input/<name>` | digest and bytes in each run, `differs` |
| `text` | `node/<node>/output/<name>/text` | with `--include-text` only: both runs' kept text of an output that differs, at most 8192 bytes each |

In it as recorded: a constant's value, whatever it holds (a path, a token), and the report's
annotations, which may name an input's folders. `--redact NAME` sends a constant's name and digests
only. Never in it: anything not in the two runs or the report, a path of this machine's own folders
(the runs', the report's, the configuration's), an environment variable, the model key, a signing
key, an anchor's proof. `--dry-run` prints the packet's bytes, their size and
sha256, and sends nothing. The first send of a kind never sent to an endpoint before names the
kinds and asks; `--yes` skips the question for scripts, the list is still printed.

## The reply, and the post-check

The model is asked for one JSON object:

```json
{"claims": [{"text": "…", "kind": "lead" | "ruled out" | "note", "cites": ["<item id>", …]}],
 "checks": [{"check": "text" | "for" | "mask" | "reproduce" | "match", "node": "…", "setting": "…",
             "run": "baseline" | "candidate"}]}
```

The post-check is mechanical, with no model. Its lists of quoting forms, look-alike letters and causal words are lists: a reply can word around them, and a kept claim is a suggestion, never a reading. A claim is dropped when its kind is not one of the
three, it cites nothing, a cite is not an item of the packet, a value it quotes (in one of the quoting forms the post-check lists) is not a value of an item it cites, or it says what caused a difference ("cause",
"because", "due to", "led to", "resulted in", "the reason", "responsible for" and their like; a list
of words, which a model can still word around: a kept claim is a suggestion, never a reading). A check is dropped when it is not on the menu, names a node or
setting the packet doesn't hold, or a run other than `baseline` or `candidate`. Drops are counted,
by reason. When every claim is dropped the explanation reads "No suggestion could be tied to the
record" and shows the checks only. A lead is confirmed or ruled out only by what a check's own
command reports, never by the model.

## What it writes

Under `OUT/explain/` (`OUT` defaults to `REPORT_DIR`), for each explanation:

- `<run id>/`: a onetrace run of its own, format 0.3, three stages: `evidence` (the report and both
  runs' manifests and receipts in, by digest; `packet.json` out), `model` (instrument: the provider
  and model name, settings: the request parameters; `reply.txt` out, the raw reply byte for byte,
  `model-generated`), `post_check` (`checked.json` out). `onetrace-verify` checks it like any run.
- `<run id>.explanation.json`: `{"label", "model", "record", "claims", "checks", "dropped",
  "checks_dropped"}`, counts as decimal strings. This is what a reader shows.
- `<run id>.explanation.txt`: the same, as the command printed it.

Every explanation is shown under the label **"Suggested explanation — AI-generated, may be wrong"**,
with its model, apart from the verdicts. Two explanations of one comparison can differ; each keeps
its own record.

<!-- https://oneproof.dev/onetrace/format.md -->

# The record format, in plain words

Full, normative detail is the Internet-Draft:
`draft/draft-saha-stage-receipts-00.txt`. This page
is an orientation, not a substitute for it.

## A receipt

One JSON file per stage a pipeline ran. Every receipt carries thirteen required members:

| Member | What it says |
|---|---|
| `format` | `stage-receipt/<major>.<minor>` — which edition of this format the receipt is written in. |
| `run_id` | Which run this receipt belongs to. |
| `stage` | The stage's own name (`retrieve`, `split`, whatever the pipeline calls it). |
| `prev` | The digest of the previous receipt in the chain, or `null` for the first. |
| `time` | `started`/`ended` timestamps, each carrying an explicit zone. |
| `coverage` | Whether this receipt's own view of the run is `complete` or `incomplete`, and which stages it declares versus which it saw emit. Since format 0.3, `optional_stages` may name declared stages the run may skip; `complete` still means every declared stage emitted. |
| `emission` | Bookkeeping about how the receipt itself was produced. |
| `anchor` | Whether the run is `anchored` (published somewhere that fixes its position in time) or `unanchored`. |
| `instrument` | What ran this stage: a name, a version, and a digest that pins its exact configuration. |
| `inputs` / `outputs` | Every artifact the stage read or wrote, each with a digest, a byte count, and a declared trust class. |
| `assertions` | Named constants the stage is asserting hold, plus values measured against them. |
| `outcome` | One of `ok`, `refused`, or `error` — never left implicit. |

A receipt is **canonical JSON**: UTF-8, no BOM, keys sorted, no incidental whitespace, no JSON
numbers (a value that looks numeric is still a decimal string — `"500.00"` and `"500"` are
different strings on purpose). The verifier checks a receipt's own bytes against its own
canonical serialization; a receipt that doesn't match its own canonical form is refused before
anything else about it is even read.

## Trust classes

Every input and output declares one of three trust classes: `operator-authored`,
`model-generated`, or `externally-sourced`. This is a claim about *provenance*, not a claim of
correctness — a `model-generated` output is not thereby untrustworthy, and an
`operator-authored` one is not thereby correct. It's what lets a reader ask "was this bit fed by
something the pipeline generated on its own, or did it come from outside?" without reading the
bytes.

## A chain

Receipts link by digest: each receipt's `prev` names the digest of the one before it, and the
manifest's own `chain_head` names the digest of the last one. A chain with a stage's receipt
missing, reordered, or altered breaks a link somewhere, and the verifier reports exactly where.

## A manifest

One `MANIFEST.json` per run, naming the chain (each entry's file and digest, in order), the
overall `chain_head`, and the run's own `format` — the manifest's format string is
`stage-receipt-chain/<major>.<minor>`, a separate version line from a receipt's own `format`. A
manifest declaring a format the verifier doesn't implement is refused outright, by name, rather
than silently accepted.

Since format 0.3 the manifest says `outputs_kept`: `"all"` (each receipt's output files are kept
beside it, under `artifacts/`) or `"none"` (the run declares it keeps none; the receipts still record
every output's digest and size). A run that keeps none also says `"outputs_kept": "none"` in every
receipt's `coverage`, under the chain and any signature. A verifier reads `"none"` as a declaration,
never as files checked, and only where the manifest and every receipt agree.

## Two roots, for a per-document stage

A stage that processes many documents in a batch (a `split` stage producing per-document chunks,
say) carries two separate digests instead of one output digest: `receipts_root` (over the
receipts' own bytes — includes timestamps, so it's never equal across two runs even of identical
work) and `outputs_root` (over document-id/output-digest pairs only — equal exactly when the
stage did the same thing to the same inputs). **Compare `outputs_root`. Never compare
`receipts_root`** for sameness — it will never agree, by design, and comparing it anyway is the
single most common way to misread this format.

## A DAG, not just a chain

A stage can declare `receipt:<node-id>` inputs naming exactly which earlier receipts it read,
turning what would otherwise be assumed as a linear sequence into an explicit graph: fan-out,
fan-in, and loops are all expressible. `receipt:` inputs are structure, not data to be
digest-compared. A node present in one run and absent in another is a difference in the shape
of the run, not something the verifier tries to explain away.

## Format 0.3, and the path to 1.0

Format 0.3 (written by onetrace 0.4.0, read in full by onetrace-verify 0.4.0) adds three members,
each designed so that a verifier that doesn't know it reads the record the same or more strictly,
never more loosely:

| Member | Where | What an earlier verifier reads |
|---|---|---|
| `coverage.optional_stages` | receipt | A run that skipped an optional stage still says `incomplete`; its `--require-complete` fails it (stricter). A run that ran every stage reads the same. |
| `refused.attempted` | receipt | The links a step held before running would have read, kept apart from `inputs`, each naming a receipt earlier in the chain with its digest. It sees a refused step and a plan the edges keep to: the same verdict, without the path. |
| `outputs_kept` | manifest; with `"none"`, every receipt's `coverage` too | `"all"` or `"none"`. With `"none"` it sees the files absent, as before: the same verdicts. onetrace-verify 0.4 reads the manifest's copy only where every receipt agrees. |

That holds for every record onetrace writes. A member written wrongly by hand (an undeclared
optional stage, `refused.attempted` on a step that ran or naming a receipt that isn't an earlier one
by its digest, an `outputs_kept` other than `all` or `none`, or receipts and the manifest that
disagree on it) is one an earlier verifier doesn't read at all, so it reads that record more loosely than
onetrace-verify 0.4, which fails it. Verify format 0.3 records with onetrace-verify 0.4.

Format 0.3's rejection vectors are in tests/rejection_0_3/: eleven runs, each changed in one of these
members and re-linked, each FAILed by onetrace-verify 0.4 on the row its INDEX.json names, with what
onetrace-verify 0.3.0 reads of it.

onetrace 0.5 stays in format 0.3: its optional blocks — `authorization` (onedoor decisions), the
span context (`trace`, `traceparent`) and the nesting fields (`parent`, `children`, `parent_run`,
`parent_stage`) — are optional members a 0.4.0 verifier reads unchanged, ignoring the blocks it does
not know, so a 0.5 receipt verifies exactly as a 0.4.0 one does and a run that records none of them
is byte-identical to 0.4.0.

The path to 1.0:

1. **0.3 is the last minor that adds a required behaviour.** After it, a 0.x minor only adds
   members an earlier verifier can ignore with the same reading.
2. **1.0 when all of these hold:** the Internet-Draft's next revision carries 0.3 as written; a
   second verifier implementation, not onetrace's own, passes the vector suite; two releases pass
   with no format change; the deprecations below are done.
3. **Before 1.0:** the preview literal `0.0-preview` stops being accepted for new records (it is
   still read); the `receipts_root` / `outputs_root` naming stays, with the warning above.
4. **What 1.0 promises:** every 1.x verifier reads every 0.x record it has ever been able to read;
   required members don't change within 1.x; a change that would change a reading is 2.0.

<!-- https://oneproof.dev/onetrace/README.md -->

# All docs pages

- [The record format, in plain words](format.md)
- [The five verdicts](verdicts.md)
- [Compare runs: diff, localize, reproduce](verbs.md)
- [What onetrace does not claim](non-claims.md)

## Worked examples with frameworks

Four pipelines, the same nine stages, the same corpus, instrumented the same way — one with no
framework at all, three wired onto a real RAG/agent framework's own runtime wherever it has one.
They are worked examples, not integrations: each is a hand-written pipeline that calls the
framework's classes inside onetrace stages, and each framework needs its own virtual environment.

- [Plain Python (no framework)](adapters/plain-python.md) — the reference path the other three are compared against.
- [LangChain](adapters/langchain.md)
- [LlamaIndex](adapters/llamaindex.md)
- [Langflow](adapters/langflow.md)

## Guides

- [Tool-using agent](tool-using-agent.md) — an agent loop recorded step by step, with a stand-in model.

See the [quickstart](quickstart.md) to run a pipeline and check
it yourself before reading further.
