Metadata-Version: 2.4
Name: tracedrift
Version: 0.1.0
Summary: Lightweight tracing and divergence diagnosis for LLM agent chains.
Author: Sandeep Eti
License: MIT
Project-URL: Homepage, https://github.com/EtiSandeep/tracedrift
Project-URL: Issues, https://github.com/EtiSandeep/tracedrift/issues
Keywords: llm,agent,tracing,observability,debugging,sqlite
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Debuggers
Classifier: Topic :: System :: Logging
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Dynamic: license-file

# tracedrift

Lightweight tracing and divergence diagnosis for LLM agent chains.

When an agent chain does something wrong, the failure usually surfaces several
steps away from its cause — a retrieval step silently returns nothing, an LLM
call improvises around it, and by the time you see the bad final answer you're
staring at raw JSON logs trying to reconstruct what happened. `tracedrift` gives
you a decorator-based tracer, a queryable SQLite trace store, a diagnosis pass
that flags *where* a chain likely diverged from its normal behavior (not just
where it crashed), and a way to replay a trace with one step's output swapped
out to see if the rest of the chain recovers.

No mandatory cloud dependency, no framework lock-in — it works with raw API
calls, LangChain, DSPy, or anything else that's just Python functions.

See [docs/USAGE_GUIDE.md](docs/USAGE_GUIDE.md) for instrumentation patterns,
how to get useful signal out of `diagnose`, async/replay caveats, and
performance notes — this README is the quickstart, that's the deep dive.

## Install

```bash
pip install -e .
```

(PyPI release TBD — this is a fresh package.)

## Quickstart

```python
import tracedrift

def retrieve(query):
    with tracedrift.span("retrieve", input={"query": query}) as s:
        s.output = my_vector_db.search(query)
    return s.output  # read back so replay overrides work — see "Replay" below

def call_llm(docs):
    with tracedrift.span("llm_call", input={"docs": docs}) as s:
        s.output = my_llm.complete(docs)
    return s.output

@tracedrift.trace_agent(name="qa_agent")
def run_agent(query):
    docs = retrieve(query)
    return call_llm(docs)

run_agent("what's our refund policy?")
```

Traces are written to `tracedrift.db` (SQLite) by default. Change it with
`tracedrift.configure("path/to/db")` before running.

## Inspecting traces

```bash
tracedrift list                    # recent traces
tracedrift show <trace_id>         # span tree with timing and status
tracedrift diagnose <trace_id>     # where did this trace likely go wrong?
```

`diagnose` doesn't just look for exceptions. It compares each span's output
against that span's own history (by name) across prior traces and flags:

1. **Explicit errors** — a span that raised. Highest confidence, reported first.
2. **Silent empty output** — a span returned `None`/`[]`/`""` when it
   historically almost never does. This catches the common case where nothing
   crashes but a step quietly failed.
3. **Type drift** — a span's output type differs from what it's always
   returned before (e.g. usually a `list`, this time a `str`).

These are intentionally simple, interpretable heuristics rather than an ML
classifier — the goal is that you can trust *why* something got flagged, not
just that it was.

## Replay with intervention

Once you suspect a step, you can rerun the same call with that step's output
forced to something else, to check whether the rest of the chain would have
recovered:

```python
result = tracedrift.replay_with_intervention(
    run_agent,
    args=("what's our refund policy?",),
    overrides={"retrieve": [<docs you think it should have found>]},
)
```

**Important caveat:** this overrides what gets *recorded* as a span's output,
not the code that runs inside the `with span(...)` block. For an override to
actually change downstream behavior, your traced function needs to read
`s.output` back out after the block (as in the quickstart above) rather than
returning a value it computed before assigning `s.output`. This is a
deliberate simplicity tradeoff — true arbitrary step-skipping would require
capturing and replaying closures, which is out of scope for a library this
size.

## Design notes

- **Storage**: SQLite by default, one file, no server. `Storage` is a small
  wrapper around raw SQL — swap it out if you need Postgres for concurrent
  writers.
- **Nesting**: spans nest automatically via `contextvars`, so this is safe
  across `asyncio` tasks (though not yet tested under heavy concurrent load —
  contributions welcome).
- **Serialization**: inputs/outputs are stored as JSON via `json.dumps(...,
  default=str)`, so non-JSON-serializable objects get stringified rather than
  crashing the tracer.

## Async

`trace_agent` and `span` both work with `async def` code — no separate API:

```python
async def retrieve(query):
    async with tracedrift.span("retrieve", input={"query": query}) as s:
        s.output = await my_vector_db.search(query)
    return s.output

@tracedrift.trace_agent(name="qa_agent")
async def run_agent(query):
    docs = await retrieve(query)
    ...
```

`trace_agent` detects `async def` functions automatically and awaits them.
Nesting relies on `contextvars`, which `asyncio` copies into each `Task` at
creation time — spans nest correctly as long as they run within the same
`Task` as the enclosing `trace_agent` call; spans started in a separately
created `Task` (e.g. via `asyncio.create_task`) won't see updates made after
that task started.

## Cost/token tracking

Record usage on a span with `s.set_usage(...)` — tracedrift doesn't know
anything about specific providers' response shapes, so pull the numbers out
yourself and pass them in:

```python
with tracedrift.span("llm_call", input=docs) as s:
    resp = my_llm.complete(docs)
    s.output = resp.text
    s.set_usage(
        prompt_tokens=resp.usage.prompt_tokens,
        completion_tokens=resp.usage.completion_tokens,
        cost_usd=resp.usage.cost_usd,
    )
```

`tracedrift show <trace_id>` prints per-span usage and a trace-level total.
Programmatically, `tracedrift.trace_usage(trace_id)` sums prompt/completion/
total tokens and cost across every span in a trace that called `set_usage`
(returns `{}` if none did).

## Web UI

```bash
tracedrift serve                 # http://127.0.0.1:8765
tracedrift serve --port 9000
```

A minimal, read-only, stdlib-only local server: trace list, span tree per
trace, diagnosis and usage totals inline. No JS framework, no build step, no
auth — it's meant for browsing your own local `tracedrift.db` during
debugging, not for exposing over a network.

## What's not here yet

- Framework-specific adapters (LangChain callback handler, DSPy module wrapper)

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md) for dev setup, project layout, and
design principles the codebase holds itself to. Please also read the
[Code of Conduct](CODE_OF_CONDUCT.md). Found a security issue? See
[SECURITY.md](SECURITY.md) instead of opening a public issue.

## License

MIT
