Metadata-Version: 2.4
Name: agent-watch-otel
Version: 0.2.0
Summary: Agent Watch's OpenTelemetry span processor: production LLM spend as digests, never prompt text.
Author: Agent Watch
License-Expression: MIT
Project-URL: Homepage, https://agent-watch.ai
Project-URL: Source, https://github.com/Agent-Watch-AI/agent-watch-edge
Keywords: opentelemetry,phoenix,openinference,llm,cost,observability
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: System :: Monitoring
Classifier: Typing :: Typed
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: opentelemetry-sdk>=1.20
Dynamic: license-file

# agent-watch-otel

**Your production LLM spend in Agent Watch, hashed in your own process, and over-budget calls refused before they leave it.**

One line beside your Phoenix registration. Prompt text, tool arguments and model responses never
leave the process — only digests, token counts and ids do. Phoenix keeps receiving everything,
unchanged.

```bash
pip install agent-watch-otel
```

```python
from phoenix.otel import register
from agent_watch_otel import AgentWatchSpanProcessor

tracer_provider = register(project_name="Production", endpoint=..., batch=True)
tracer_provider.add_span_processor(AgentWatchSpanProcessor(token="..."))  # beside yours
```

The token is an Agent Watch SDK token scoped `ingest:production`. `token` and `endpoint` may also
come from `AGENT_WATCH_TOKEN` and `AGENT_WATCH_ENDPOINT`.

---

## What it sends

Finished spans whose `openinference.span.kind` is `LLM`, and nothing else. Per call:

| Sent | Not sent |
|---|---|
| sha256 of the system/developer prefix, and its 8-token window digests | the prompt |
| sha256 of the tool definitions | the tool definitions, and every tool argument |
| model, provider, token counts (input, output, cache read/write, reasoning) | the response |
| span id, trace id, agent name (see "Which agent"), end time | anything the span carries that is not in this table |
| whether the call failed and, if it reached the provider, an input estimate from its request body's size | the request body |
| `service.name` and `service.version` from the OTel resource | |

Before a model call, the gate sends a decision request (see "Refusing over-budget calls"). It
carries the agent name (see "Which agent"), `service.name`, the model and the deployment environment, plus counts of
local refusals and fail-opens. Never prompt text, tool definitions or responses.

That last row is why this package exists rather than only the poller: **Phoenix drops resource
attributes at ingest**, so a span processor is the only thing that can report which service and
which version produced a call.

Digests are computed in `on_end`, on your thread, before anything is queued. The queue holds
hashes; the sender thread never sees a prompt.

## What it does to your application

One thing in it can stop a call: an over-budget refusal (see "Refusing over-budget calls"). Nothing
else in the package reaches your application.

- **`on_end` never touches the network.** It hashes and enqueues. A background daemon thread
  batches and POSTs.
- **`on_end` never raises.** Every path out of it is wrapped. A bug here costs you a cost record,
  never a request.
- **The queue is bounded.** When Agent Watch is unreachable or slow, calls are dropped and counted
  rather than buffered without limit inside your process.
- **It survives `fork()`.** Under gunicorn `--preload`, uWSGI or Celery prefork, each child gets
  its own sender thread rather than silently dropping every call.
- **Shutting down is bounded.** An endpoint that accepts the connection and never answers cannot
  stretch `shutdown()` past `shutdown_seconds`, however many times it is called — so stopping
  your process stays inside a short `SIGTERM` grace period. Call `force_flush()` instead when
  what you want is delivery rather than a prompt exit.

```python
processor.stats()
# {'sent': 412, 'dropped_queue_full': 0, 'dropped_send_failed': 3,
#  'dropped_no_model': 0, 'dropped_error': 0}
processor.dropped   # 3
```

A call with no `llm.model_name`, `llm.model` or `gen_ai.request.model` is counted under
`dropped_no_model` rather than sent under a guessed model, because spend priced against the wrong
model is worse than spend that is visibly missing. A refused call has no spend, so it is sent with
the model `unknown` instead.

## Refusing over-budget calls

When a feature's production budget is spent, the agent's next model call is refused before it
leaves your process. If `httpx` is installed (the OpenAI and Anthropic SDKs use it), registering
the processor wraps `httpx.Client.send` and `httpx.AsyncClient.send`. The first request made inside
an LLM span (`openinference.span.kind = LLM`, the spans this processor reports) asks Agent Watch
about the agent the span names. Tool, agent and embedding spans, and spans that only set
`gen_ai.operation.name`, are never gated. A refused call raises:

```
AgentWatchRefusal: An Agent Watch budget refused this call.
```

It names no cap, amount, feature or policy, and `retryable` is `False`: a retry gets the same
answer from the held decision, with no request to Agent Watch or to the provider. Provider SDKs
raise their own connection error with the refusal as its `__cause__`, and may retry with their own
backoff first. Test for it with `is_agent_watch_refusal`:

```python
from agent_watch_otel import is_agent_watch_refusal

try:
    client.chat.completions.create(**request)
except Exception as err:
    if is_agent_watch_refusal(err):
        return budget_spent_answer()
    raise
```

**The cost.** Answers are held per agent and deployment environment for as long as Agent Watch
says (at most 60 s); a held answer adds no request. A fresh one waits at most 300 ms, with one
request per agent in flight at a time, shared by threads and event loops: concurrent calls for the
same agent wait on that one request, and every request in one span shares the span's decision. The
300 ms covers resolving the host too. A request still unanswered at 300 ms is not cut off: the call
goes ahead as a fail-open, the request runs on in its daemon thread (at most 5 s, never holding the
process open), and its answer is held for the agent's next calls, unless a newer request was sent
meanwhile. So a gateway far away, slower than 300 ms on a fresh connection, still gets to refuse. A
thread still alive past those 5 s (a resolver that hangs cannot be interrupted) counts as hung: no
second one starts for that agent while it lives, and once four are hung, or 64 decision threads are
alive at all, no new request starts until one ends. Many agents asking at once are all asked. An async wait never blocks the event loop, and a cancellation while it waits propagates at once.

**Which agent.** The agent is the first of the span attributes `agent.name`, `gen_ai.agent.name`
and `llm.task` that is set and not blank, else the span name: the same rule Agent Watch uses when
it reads your spans from Phoenix or Langfuse, so both name one agent. OpenInference's OpenAI
instrumentation names every span `ChatCompletion`, so set `agent.name` (or wrap the call in a span
named for the agent), or all such calls are one agent.

**Fail-open, always.** If Agent Watch is slow, unreachable, answers anything but 2xx, or answers
something it cannot read, the call goes ahead, and the fail-open is reported on the next decision
request for that agent. What is owed travels as one batch with a `report_id`, resent unchanged
until a request carrying it is answered, so Agent Watch records it once. After a fail-open the agent is allowed without asking for 30 s, or for
5 minutes after a 401, 403 or 404 (the route is missing or the token lacks the scope); each call
let through meanwhile is counted as a fail-open too. There is no fail-closed mode and no switch
for one. Nothing inside the package reaches your application as anything but the refusal.

**What passes ungated.** Requests outside an LLM span; requests to `metadata.google.internal`,
`oauth2.googleapis.com` and `sts.googleapis.com`, so a workload-identity token fetch is never
refused; and the package's own requests. A refused call is still reported, with `refused: true`,
and counts no spend; when its span names no model it is reported as model `unknown`.

**A call that failed.** A span ended with an error status is reported with `errored: true`. When
the `httpx` request reached the provider and failed (a 5xx status, or a timeout or broken
connection after it was sent; a 4xx such as 429 is not billed and is not estimated), the package records the request body's length on the span and reports the input as that
length over 4, rounded up, with `input_estimated: true` and output 0; the provider's own count wins
when the span has one. A DNS or connection failure never reached the provider and carries no
estimate. Only the length is read, never the body, and nothing is measured on a call that succeeds.

**Known limit: sampled-out spans are never gated.** A span your sampler drops carries no name or
attributes, so a call inside it goes ahead unjudged and is not counted anywhere. Keep LLM spans
sampled wherever you rely on the gate. The span must also be current on the thread or task making
the request, which is where instrumentation puts it.

**Other transports.** `requests`, `urllib`, gRPC and other clients are not intercepted. Call
`guard(agent_name)` (or `await guard_async(agent_name)` in a coroutine) before such a call; it raises `AgentWatchRefusal` when refused, and returns at
once when no processor is registered. It states the agent and the deployment environment from
`OTEL_RESOURCE_ATTRIBUTES`; with neither environment key set there, it is judged as production.

**Dev and CI.** Set the resource attribute `deployment.environment.name` (or the older
`deployment.environment`, read when the first is unset) to anything but `production` or `prod`
and calls are allowed without judging; unset, they are judged as production.

**Behind a proxy.** Requests to Agent Watch, ingest and decisions alike, honour `HTTPS_PROXY`,
`HTTP_PROXY` and `NO_PROXY` from the environment (lower case too). An https endpoint is
tunnelled with CONNECT, so TLS still ends at Agent Watch; credentials in the proxy URL are sent
to the proxy only. Redirects are never followed, proxy or not.

## Options

| Argument | Default | What it is |
|---|---|---|
| `token` | `$AGENT_WATCH_TOKEN` | SDK token scoped `ingest:production` |
| `endpoint` | `$AGENT_WATCH_ENDPOINT` | Ingest base URL; the route is `v1/runtime/calls` |
| `instance_id` | the resource's `service.name` | This deployment's name, as you know it |
| `max_queue` | 2048 | Calls held while the sender is behind |
| `flush_seconds` | 5.0 | How long a partial batch waits |
| `timeout_seconds` | 10.0 | Network timeout for one POST |
| `shutdown_seconds` | 3.0 | The whole budget `shutdown()` may spend, across every call |

A missing token or endpoint raises at construction — on your startup path, where a misconfiguration
is visible — never later from `on_end`.

## Requirements

Python 3.9+ and `opentelemetry-sdk`. Nothing else, by policy: this code runs inside your
application, so every dependency it took would become a version you have to resolve. `httpx` is
gated when it is installed and never required.

## Development

```sh
python -m venv .venv && .venv/bin/pip install opentelemetry-sdk
.venv/bin/python -m unittest              # the whole suite
.venv/bin/python -m unittest tests.test_vectors   # the fingerprint parity check
.venv/bin/pip install httpx && .venv/bin/python -m unittest tests.test_gate_conformance   # the gate parity check
```

`conformance/runtime/gate-cases.json` is the gate's behaviour (gateway answers in, outcomes out),
replayed by `tests/test_gate_conformance.py` with sync and async `httpx`, and by the Node package.
The suite runs without `httpx` too; the tests that need it are skipped.

`conformance/runtime/fingerprint-vectors.json` (at the repository root) is a byte copy of core's
golden fingerprint vectors (`packages/fingerprint/test/vectors.json`, AWT-263). `tests/test_vectors.py`
replays every case through this port, and the Node package replays the same file, so the
implementations cannot drift apart without CI going red. It also covers a case no shared vector can
express — see `MAX_TOOL_DEPTH` in `agent_watch_otel/fingerprint.py`.

## Licence

MIT.
