Metadata-Version: 2.4
Name: onetrace-auto
Version: 0.9.0
Summary: Zero-configuration recording for onetrace: a run folder from a library's calls, with nothing added to the app
Author-email: Shamik Saha <shamik.saha.rcciit@gmail.com>
License-Expression: Apache-2.0
Project-URL: Homepage, https://oneproof.dev/onetrace/
Project-URL: Issues, https://github.com/shamiksaharcciit-oss/onetrace-issues
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: onetrace==0.9.0

# onetrace-auto

Zero-configuration recording for [onetrace](https://pypi.org/project/onetrace/): run an app as
you run it today, with one thing added at the edge, and get a run folder with a receipt per hooked
library call. Released with onetrace, at the same version, and pinned to it.

## Install

```
pip install onetrace-auto
```

This installs one `.pth` line into the environment. It runs at every interpreter start there, and
with `ONETRACE_AUTO` unset it reads two environment variables and imports nothing. The `onetrace`
package alone installs no `.pth` line.

## Use

```
onetrace-run python app.py
```

or set the variable yourself:

```
ONETRACE_AUTO=1 python app.py
```

or turn the hooks on in your own code, at its start:

```python
import onetrace.auto

onetrace.auto.instrument()
```

Only the value `1` turns recording on from the variable, and `ONETRACE_DISABLE=1` turns it off whatever else is set.
`onetrace-run` and `ONETRACE_AUTO=1` call `onetrace.auto.instrument()` for you, at interpreter start.

`onetrace.auto.instrument()` turns the hooks on whatever `ONETRACE_AUTO` holds, and returns the hook list the run's
manifest names in `auto.hooks`. Its keywords stand in for the variables, and one left out reads its variable:
`runs=` for `ONETRACE_RUNS`, `per=` (`"process"` or `"request"`) for `ONETRACE_AUTO_PER`, `keep=` (a list of
`"inputs"`, `"outputs"`) for `ONETRACE_AUTO_KEEP`, `propagate=` (a list of exact hosts) for
`ONETRACE_PROPAGATE`, and `sample=` (a rate as text, such as `"0.25"`) for `ONETRACE_AUTO_SAMPLE`. An argument out of form raises `ValueError`, naming it. A second call with the same settings
does nothing; one with other settings raises `ValueError`, naming the setting that differs, the settings of
`onetrace-run`'s own call included. Under `ONETRACE_DISABLE=1`, and in the child `onetrace check` starts, it does
nothing and returns `[]`, so a pipeline's own call records no hooked step in a check. In the child `onetrace
reproduce` starts (a reproduce, a twin or a counterfactual), it turns the hooks on, and the re-run's replay rule
applies: a hooked call the plan stops before raises `RerunStopped`, and the library is not called. A re-run plan
onetrace refuses makes the call raise `EmissionRefused`. Where the hooks can't start, it says so in one line on
stderr, returns `[]`, and your program goes on.

Called in your code, the hooks start where the call stands:
- **Call it before you make any client.** A library imported before the call is hooked at the call, but a client
  made before it is recorded only where its hook can see it: an openai client made before it writes no model-call
  record, and an S3 client made before it is not seen at all. Nothing warns you of a client made earlier.
- **An exit handler your program registered before the call runs after the run's close,** since Python runs exit
  handlers last registered first; a hooked call made in it is not recorded.
- **SIGTERM:** the run's handler is installed only where your program has none when the call is made, and only by a
  call made in the main thread.
- **A child process `multiprocessing` starts with `spawn`** records only where its own code reaches the call. Under
  `onetrace-run` it inherits `ONETRACE_AUTO=1` and turns the hooks on at its start; a forked child keeps the hooks
  and records its own run.
Each Python process records its own run, under `ONETRACE_RUNS` (default `./runs`), opened at its
first hooked call: a process that makes none leaves no folder.

`onetrace-run --runs DIR` sets `ONETRACE_RUNS`. `onetrace-run --per process` sets `ONETRACE_AUTO_PER=process`,
one run per process, and `onetrace-run --per request` sets `ONETRACE_AUTO_PER=request`. A flag given wins over the
variable `onetrace-run` inherited, as a keyword of `instrument()` does; one left out leaves the variable as it is.

### A run per request: FastAPI and Flask

```
onetrace-run --per request uvicorn app:app
onetrace-run --per request flask --app app run
```

With `--per request` (or `ONETRACE_AUTO_PER=request`, or `instrument(per="request")`), each HTTP request a FastAPI or
Starlette app (starlette `>=0.37,<2`) or a Flask app (flask `>=2.3,<4`) serves gets its own run folder, opened when
the request comes in and closed when its response is complete. A hooked call made while the request is served
records into that run. The run's manifest holds `auto.boundary` `request`, and `auto.key` what keyed it:
- the request's `traceparent`, where it is well formed, as `<trace id>-<parent span id>`;
- else its `X-Request-ID`, where that value could name a run folder;
- else the run's own generated id.

The header is read to key the run; it is not recorded as a header. The request's method, route and path are not
recorded. A request that sent `onetrace-parent` starts its run with that parent, and its response gets
`onetrace-child`. A Flask body produced after `wsgi_app` returns, a lifespan or websocket scope, and a hooked call
made outside any request record into the process run.

An auto-recorded run is complete only with respect to the libraries hooked, at the versions its
manifest lists in `auto.hooks`. A step in your own code is recorded only where a decorator names
it. A run folder may contain sensitive data: by default it holds digests only.

## Hooked libraries

| library | versions | stage | what a step records |
|---|---|---|---|
| openai | `>=1.40,<4` | `openai.chat.completions.create`, `openai.responses.create`, `openai.embeddings.create`, `openai.completions.create` | `create` on each, on `OpenAI` and `AsyncOpenAI`; a stage is declared only where the installed version has its resource (1.40 has no `responses`) |
| requests | `>=2.28,<3` | `requests.send` | `Session.send`, so `requests.get(...)`, every session method and a prepared request sent by hand |
| httpx | `>=0.24,<1` | `httpx.send` | `Client.send` and `AsyncClient.send`, so every request method |
| anthropic | `>=2,<3` | `anthropic.messages.create` | `create` on `Messages` and `AsyncMessages`, and the stream helper `messages.stream` on each, a step of the same stage with `auto.call` `messages.stream`; `messages.batches` and `beta.messages` are not hooked |
| litellm | `>=1.80,<2` | `litellm.completion` | the functions `litellm.completion` and `litellm.acompletion`, sync, async and streamed, and `completion_with_retries` and `acompletion_with_retries`, `auto.call` the function's name; read through litellm's `CustomLogger` callbacks; a fallback (to another model, to the same model with another body, or to another host or path) and a redirect write no model-call record; a Bedrock call writes a record only where boto3 is installed, its `model.reported` always `not observed`; `text_completion`, `embedding` and `responses` are not hooked |
| google-genai | `>=1.50,<4` | `google_genai.models.generate_content` | `generate_content` and `generate_content_stream` on `Models` and `AsyncModels` (`auto.call` `models.…` or `aio.models.…`); a chat's `send_message` is recorded as the `generate_content` it calls |
| langgraph-checkpoint (LangGraph's store) | `>=2,<5` | none: an entry on the step open now | `get`, `put`, `delete`, `search` and `list_namespaces` on `BaseStore`, and their async forms, as `state_reads` and `state_writes` entries, `store_kind` `langgraph`; with no step open, nothing |
| mem0ai | `>=1,<3` | none: an entry on the step open now | `add`, `get`, `get_all`, `search`, `update` and `delete` on `Memory` and `AsyncMemory`, as `state_reads` and `state_writes` entries, `store_kind` `other`; mem0's own model calls are the openai hook's steps; with no step open, nothing |
| letta-client | `>=1,<2` | none: an entry on the step open now | memory blocks (`blocks` and `agents.blocks`) and archival passages (`archives.passages`, and from 1.12 `agents.passages` and `passages.search`), as `state_reads` and `state_writes` entries, `store_kind` `other`; the request beneath is the httpx hook's step; with no step open, nothing |
| zep-cloud | `>=3,<5` | none: an entry on the step open now | a thread's messages and context (`zep:threads`) and the graph's episodes, searches and context (`zep:graph`), sync and async, as `state_reads` and `state_writes` entries, `store_kind` `other`; the request beneath is the httpx hook's step; with no step open, nothing |

An HTTP step records the request body as its input `request`, and two outputs: `status`, the JSON
object `{"status": "<code>"}`, and `response`, the body's bytes. A requests call that followed
redirects is one step: its `status` is the final response's, with the number of hops,
`{"redirects": "<n>", "status": "<code>"}`, and its `response` is the final body. A requests text body is
digested as the installed urllib3 sends it: ISO-8859-1 below urllib3 2, UTF-8 from 2. Where
ISO-8859-1 cannot encode it, the step records no `request` input and says why. A streamed response's body read
to its end is hashed as the app reads it, decoded, and its `response` entry names `content_encoding`: the header as
received where the library undid an encoding, `identity` where it undid none. One closed before its end records
`status` alone and says so in the constant `onetrace.http.body`. A
streamed request body is not read either: the step has no `request` input and says so in
`onetrace.http.request`. The target is the host, the method, the path's sha256 and the query's
parameter names; no query value and no user info is recorded. An error records its class, never its
message, since an HTTP library's message can hold the URL.

A model call records its request body as `request`, the response's `model_dump()` as `response`,
and a model-call record, `model_call.json`, by the hosted adapter's rules. The record's
`credential` (only whether a key is set, never the key) and the step's host come from the client's
own `post` method, read while the call runs. A streamed call returns the library's stream, wrapped:
the step ends when the stream does, and a stream left before its end is an error. Usage is in
`response`, not in the record.

**anthropic's stream helper.** `with client.messages.stream(...) as stream:` is one step, opened when the `with` block
is entered and recorded as `client.messages.create(..., stream=True)` with the same arguments is: the same `request`,
the stream's events as `response`, and the record read from them, which make up the final message: the model from its
start, the usage from its start and its last delta, the stop reason and the text from the deltas. What the stream never
reports, such as a model or an input count it does not send, reads `not observed`. The step ends when the stream has
been read to its end (`get_final_message()`, `until_done()`, `text_stream` or the events), is an error with the
exception's class when the stream raises, and is an error, `stream not consumed to its end`, when the block is left
before the end. An error records its class and its message, with every URL's user
info and query values removed.

**litellm.** Each `litellm.completion` and `litellm.acompletion` call is one model-call step. The hook adds its own
`CustomLogger` to `litellm.callbacks` when litellm is imported, and reads through it the provider's own response, as
litellm hands it on. Where litellm calls the provider through the openai package, the litellm step records the call:
the openai call and the HTTP request beneath it are no steps of their own. What the step records:
- **the request:** the request's own arguments (`model`, `messages`, `temperature`, `max_tokens`, `tools` and the
  rest litellm documents for `completion`). `api_key`, `api_base`, `client`, `headers` and every other argument are
  left out, and their names too;
- **the record's host, path and `credential`** come from the last HTTP request litellm sent under the call, as the
  httpx, httpx2 or requests hook sees it pass: whether it held an `Authorization`, `x-api-key` or `api-key` field,
  never its value. A new request drops the provider's body seen before it, so a record's request and body are one
  attempt's.
  Where no such request was seen, as where the call failed before one went, the step writes no record and the constant
  `onetrace.model_call` says so;
- **`model.requested`** is the model the call asked for (`openai/gpt-4o-mini`, say). **`model.reported`** is the
  model the provider's own response names. litellm puts the name asked for into its `ModelResponse.model` and into
  every stream chunk it yields, so neither is read as the provider's;
- **`model_usage`** is the provider's own counts, from its response. litellm's `usage` is not read: litellm counts
  tokens itself where the provider sends none;
- **a stream through the openai package** is read from the provider's own chunks: litellm reads the openai package's
  stream, and the hook reads each chunk as it passes to litellm, unchanged. `model.reported` is the model the chunks
  name, and `model_usage` the counts in the last chunk that holds any (litellm asks openai for them on a stream).
  **A stream litellm parses itself** (its own HTTP handler, as for its anthropic provider) shows no provider chunk:
  its `model.reported` and counts read `not observed`;
- **no cost:** litellm's `response_cost` and `completion_cost()` come from a price list, not from the provider, and
  are not written. `model_usage` has no `cost`.

**Retries, fallbacks and the Router.** A retry of the same model (`num_retries`, `completion_with_retries`, or the
openai package's `max_retries`) stays in the call's one step, and its record is the last attempt's. A `Router` calls
litellm once per deployment it tries, so each attempt is its own step with its own record. A fallback inside one call
(`fallbacks=`) asks for another model, or for the same model with other arguments. Where litellm names more than one
model or provider for the call's attempts, or the HTTP requests under the call send more than one body or go to more
than one host and path, the step writes **no** model-call record, no `model_usage`, no `model_call_failed` and no
`target`: the arguments the call was given and the body that answered come from different attempts. The hook compares
the bodies by their sha256 and keeps no body. A fallback to another base URL with the same model and body counts by
its host and path. So does a redirect, read from the responses the HTTP library hands back: its record is left out
too, and the step does not say where the answer came from. A retry that sends the same body to the same place again is
no fallback by this rule. The step's `request` and `response` are still recorded. It carries no `onetrace.model_call`
constant either: the format gives that constant two values, and neither says what happened here. A third value was
tried, `not written: a fallback asked for more than one model`: onetrace-verify 0.6.1 reads such a run exit 0, but
0.7.0, 0.7.1 and 0.8.0 read any value outside their list as a FAIL, exit 1. So the record is left out with nothing
written in its place.

litellm's sync retry paths (`num_retries` on `completion`, `completion_with_retries`, and
`acompletion_with_retries`, which runs the sync `completion`) import `tenacity`, which litellm's own install does not
bring. Those paths need it installed: that is litellm's requirement at that point, not onetrace-auto's. The test
venvs pin it by hash so that those paths are tested.

**Bedrock through litellm.** litellm signs a Bedrock request with boto3 and sends it through its own HTTP handler,
which the httpx hook sees. Where boto3 is installed, the step writes a record: the host
(`bedrock-runtime.<region>.amazonaws.com`), the path (`model/<model id>/converse`), `credential` present (the
signed request's `Authorization` field, never its value), and the Converse response's own counts (`inputTokens`,
`outputTokens`, `totalTokens`). Bedrock's response names no model, so **`model.reported` always reads `not
observed`**. Where boto3 is not installed (litellm's own install does not always bring it), litellm raises before any
request goes: the step is an error with **no record**, and the constant `onetrace.model_call` says so.

A stream left before its end is an error, sync or async, as for the other model hooks. A call that fails after its
HTTP request went writes a `/1.1` record and `model_call_failed`, with the source `the call failed after its HTTP
request went`. onetrace-verify documents no
effect-label bases for `litellm`: its row reads NOT-RUN, `litellm: no documented bases`, and the exit is unchanged.
**litellm's price list.** Unless `LITELLM_LOCAL_MODEL_COST_MAP=True` is set, litellm fetches its price list when it
is imported, with `httpx.get`. That fetch is no model call and makes no litellm step, but the httpx hook records it:
an `httpx.send` step, a `GET` to the price list's host (`raw.githubusercontent.com` by default), labelled `read`. So a
program that imports litellm has that step in its run even if it never calls a model. With the variable set to
`True`, litellm reads its own copy and nothing is fetched or recorded. A stream from `completion` read with `async for`
(litellm allows it) is recorded as when it is read with `for`.
**google-genai.** Each call of `generate_content` or `generate_content_stream`, sync or async, is one model-call
step. Its HTTP request, through httpx (google-genai below 3) or httpx2 (from 3), passes straight through under it:
the request gives the record its path as `call.api`, its host as `auto.target.host`, and as `credential` whether it held
an `x-goog-api-key` or `Authorization` field (never its value). The step's `request` is the call's own arguments,
`model`, `contents` and `config` without `http_options`; `model.requested` is the `model` asked for, and
`model.reported` the response's `modelVersion`, or `not observed` where it gives none. `model_usage` is the response's
usage metadata (`promptTokenCount`, `candidatesTokenCount`, `totalTokenCount`), written only with the record. A stream
is google-genai's generator, wrapped: its request is sent at the first read, and the step is open while it is read; it
ends when the stream is read to its end, raises, or is closed or collected before its end (`stream not consumed to its
end`, with no record). A call that sends more than one request body, as automatic function calling does with a Python
function given as a tool, is one step with no record, no `model_usage` and no `target`. An async call google-genai
sends through aiohttp, where aiohttp is installed, is a step with no record: onetrace-auto hooks no aiohttp.
**LangGraph's store.** A store hook opens no step: each call of a `BaseStore` method adds one entry to the step open
now, the auto step running or a decorated stage, as `ot.state.read` and `write` do. `get`, `search` and
`list_namespaces` add a read, `put` and `delete` a write, each with `store` `langgraph:<store class>`. The key is the
namespace and the key by its digest (a search's, what it asked; a listing's, its conditions); the value is digested as
canonical JSON (`json/1`): what `put` wrote, the item `get` found, a search's items, a listing's namespaces. A `get`
that finds nothing reads `found` `false`, and a `delete` says it writes no value. A `get`'s version is the item's
`updated_at` (`version_kind` `updated_at`); every other entry's is `not observed`, and a search's `as_of` is its items'
newest and oldest `updated_at`. A call that raises adds no entry. A call made through `batch` or `abatch` directly is
not hooked.
**mem0.** A store hook opens no step: each call of `add`, `get`, `get_all`, `search`, `update` or `delete` on
mem0's `Memory` or `AsyncMemory` adds one entry to the step open now, `store` `mem0:<class>`, `store_kind` `other`
(the format's list of store kinds has no `mem0`; onetrace-verify 0.8.0 FAILs a kind outside it). The key is the user,
agent and run ids with the memory id by its digest (mem0 2 takes the ids inside `filters`, mem0 1 beside them); the
value is digested as canonical JSON (`json/1`). A `get` that finds nothing reads `found` `false`, a `delete` says it
writes no value, and a `get`'s version is the memory's `updated_at` where mem0 gives one. mem0's own model and
embedding calls go through its provider: through the openai package, each is the openai hook's step, one per
request, and the mem0 hook adds none. mem0 1 makes `add`'s and `search`'s calls on worker threads of its own, so those
steps are in onetrace-auto's own run, beside a decorated run the mem0 call was made in.
mem0 1.0.0 itself writes a local database under `~/.mem0` even with its telemetry off; `MEM0_DIR` moves it. onetrace
neither causes nor changes this.
**Letta.** A store hook opens no step: each call on letta-client's memory blocks (`blocks.create`, `retrieve`,
`update`, `list` and `delete`; `agents.blocks.retrieve`, `update`, `list`, `attach` and `detach`) or archival passages
(`archives.passages.create`, `create_many` and `delete`; from 1.12, `agents.passages.create`, `list`, `search` and
`delete`, and `passages.search`) adds one entry to the step open now, `store` `letta:blocks` or `letta:passages`,
`store_kind` `other` (the format's list of store kinds has no `letta`; onetrace-verify 0.8.0 FAILs a kind outside it).
The key is the store's own ids by its digest, with a listing's or a search's arguments; the value is what the server
returned, digested as canonical JSON (`json/1`); a `delete` says it writes no value, and an `attach` or a `detach`
writes the link. The version is a passage's `updated_at` where one passage is returned; a block carries none. The
HTTP request beneath each call is one `httpx.send` step, as any httpx request is. The async client's `list` methods,
which return a paginator, and the pages fetched after a listing's first are not recorded. Letta's own model calls are
the server's, and no hook here sees them.
**Zep.** A store hook opens no step: each call on zep-cloud's threads (`thread.add_messages` and `delete`; in 3
`thread.get` and `get_user_context`, in 4 `thread.list_messages` and `get_context`) or graph (`graph.add` in 3,
`graph.episode.add` in 4, `graph.episode.get` and `delete`; `graph.search` in 3; `graph.search_edges`, `search_nodes`,
`search_episodes` and `get_context` in 4) adds one entry to the step open now, `store` `zep:threads` or `zep:graph`,
`store_kind` `other` (the format's list of store kinds has no `zep`; onetrace-verify 0.8.0 FAILs a kind outside it).
The key is the store's own ids, named as the installed client names them, by its digest, with a read's other
arguments; the value is the messages given for an add, else what the server returned, in the client's own form,
digested as canonical JSON (`json/1`), a paged read's first page; a `delete` says it writes no value. The version is
`not observed`: Zep returns none. The HTTP request beneath each call is one `httpx.send` step, as any httpx request
is. Zep's own model calls are its service's, and no hook here sees them.

What a model call does **not** record:
- **where openai was imported before onetrace-auto's boot,** a call on a client made then records
  no credential, no host and no model-call record. The step is recorded, and the constant
  `onetrace.model_call` says `not written: the hook did not see the client's request`. With
  `onetrace-run`, or `ONETRACE_AUTO=1` set before the interpreter starts, boot comes first; with
  `onetrace.auto.instrument()` in your code, call it before you make the client;
- **an embeddings call** has no record format in this release: it records `request` and
  `response`, and the constant says so;
- **under openai 3.x, the client's HTTP layer** (`httpx2`) is not recorded by the httpx hook. The
  model step itself is recorded.

## Settings

- `ONETRACE_AUTO_KEEP=outputs` keeps the output files; `inputs` keeps each request body of up to
  4096 bytes (a larger one is named, not kept). Both: `outputs,inputs`.
- `ONETRACE_AUTO_MAX_RUN_BYTES=<bytes>` gives the process's run a store bound, in decimal digits: the
  receipts and kept outputs count toward it, the manifest does not. The step that reaches it is
  recorded as a gap, the run declares the boundary `store bound reached` naming that step and
  closes, and every later hooked call goes ahead unrecorded. Unset or empty: no bound. A value that
  is not a whole decimal number, or is below the smallest the run's receipts allow, opens no run,
  and the process says why once on stderr. It is read once, when the run opens; a decorated
  `@ot.run` does not read it.
- `ONETRACE_AUTO_SAMPLE=<rate>`, a rate from 0 to 1 in decimal digits (`0.25`), records that share of runs. Each
  run that would open is decided once, when it would open, and never mid-run: the process's run at its first hooked
  call, and with `--per request` each request when it comes in. A run sampled out writes nothing, no folder and no
  journal, and its hooked calls go ahead unrecorded; a request sampled out records nowhere, not into the process's
  run either. `0` records no run and `1` every run; unset or empty, every run, as before. The draw comes from
  Python's `random`, seeded from the OS. A value out of that form opens no run, and the process says so once on
  stderr, naming the variable; `instrument(sample=...)` out of form raises `ValueError`.
  - **What a sampled set does not show:** anything about the runs not recorded. A set recorded at `0.1` is the runs
    the draw let in, and says nothing of what the others did, whether they failed, or how many there were.
  - **Written into the run:** a sampled-in run, the process's or a request's, carries the rate as given, a string,
    in its manifest's top-level member `auto_sample` (`"0.25"`), not inside `auto`. A run with no rate set carries
    none, and so does a subgraph's nested run, which is not drawn. onetrace-verify 0.6.1, 0.7.0, 0.7.1 and 0.8.0
    each read a run with the member exit 0, with no row about it.
- `ONETRACE_AUTO_HTTP_PATH=plain` records the path itself in place of its digest.
- `ONETRACE_AUTO_KEEP_QUERY=name,...` records the named query parameters' values as configuration.
