Metadata-Version: 2.4
Name: agent-rewind
Version: 0.8.0
Summary: Record, replay, and time-travel-debug LLM agent runs.
Project-URL: Homepage, https://github.com/Abhi-2526/agent-rewind
Author-email: Abhinav Sood <abhinavsood401@gmail.com>
License: MIT
License-File: LICENSE
Keywords: agents,debugging,llm,replay,testing,tracing
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Debuggers
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.10
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.40; extra == 'anthropic'
Provides-Extra: cohere
Requires-Dist: cohere>=6.0; extra == 'cohere'
Provides-Extra: dev
Requires-Dist: httpx; extra == 'dev'
Requires-Dist: hypothesis>=6; extra == 'dev'
Requires-Dist: mypy; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Requires-Dist: ruff; extra == 'dev'
Provides-Extra: e2e
Requires-Dist: langchain-fireworks; extra == 'e2e'
Requires-Dist: langchain-groq; extra == 'e2e'
Requires-Dist: langchain-openai; extra == 'e2e'
Requires-Dist: langgraph; extra == 'e2e'
Requires-Dist: openai-agents; extra == 'e2e'
Requires-Dist: pydantic-ai; extra == 'e2e'
Provides-Extra: embeddings
Requires-Dist: sentence-transformers>=2.2; extra == 'embeddings'
Provides-Extra: fireworks
Requires-Dist: fireworks-ai>=1.2.0; extra == 'fireworks'
Provides-Extra: gemini
Requires-Dist: google-genai>=1.50; extra == 'gemini'
Provides-Extra: groq
Requires-Dist: groq>=0.37; extra == 'groq'
Provides-Extra: lint
Requires-Dist: mypy<3,>=2.3; extra == 'lint'
Requires-Dist: ruff<0.17,>=0.16; extra == 'lint'
Provides-Extra: litellm
Requires-Dist: litellm>=1.90; extra == 'litellm'
Provides-Extra: mcp
Requires-Dist: mcp>=1.0; extra == 'mcp'
Provides-Extra: mistral
Requires-Dist: mistralai>=1.0; extra == 'mistral'
Provides-Extra: openai
Requires-Dist: openai>=1.40; extra == 'openai'
Provides-Extra: together
Requires-Dist: together>=2.0; extra == 'together'
Description-Content-Type: text/markdown

# rewind

[![CI](https://github.com/Abhi-2526/agent-rewind/actions/workflows/ci.yml/badge.svg)](https://github.com/Abhi-2526/agent-rewind/actions/workflows/ci.yml)
[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

**Record, replay, and time-travel-debug LLM agent runs.**

Agents fail at step 37 of 40 and all you have is logs. rewind records every
LLM and tool call of a run to a portable trace file, replays the run
deterministically offline (no API key, $0), lets you *fork* a run at any step
to test an intervention, and turns recorded traces into free CI tests.

- **Zero dependencies.** The core is pure stdlib — provider SDKs are optional extras.
- **Zero code changes.** `auto_patch()` hooks OpenAI, Anthropic, litellm, Mistral, Cohere, Gemini, Groq, Together AI, Fireworks, and MCP directly — works unmodified with LangChain, LangGraph, the OpenAI Agents SDK, pydantic-ai, smolagents, and anything else built on them.
- **Crash-safe.** Events are flushed per call; a crashed run still leaves a usable trace.
- **Divergence as a feature.** When a replayed run escapes its trace, the error tells you the exact step, with a diff.

## Install

```
pip install agent-rewind        # import name: rewind
```

Zero runtime dependencies. Python 3.10+.

## Quickstart

```python
import rewind

@rewind.llm
def call_model(**request):
    return client.chat.completions.create(**request)

@rewind.tool
def search(query: str) -> dict:
    return search_api(query)

# 1. Record a live run
with rewind.record("run.rewind"):
    agent.run("find me a flight")

# 2. Replay it -- fully offline, deterministic, free
with rewind.replay("run.rewind"):
    agent.run("find me a flight")

# 3. Time-travel: history up to step 23, live (with your fix) after
with rewind.fork("run.rewind", at=23, save_as="fixed.rewind") as session:
    agent.run("find me a flight")
print(session.new_events)  # what happened after the intervention
```

Don't want decorators? Patch the SDK itself instead — no code changes at all:

```python
import rewind
rewind.auto_patch()              # patches every installed provider SDK

with rewind.record("run.rewind"):
    agent.run("task")            # unmodified framework agent -- any provider

with rewind.replay("run.rewind"):
    agent.run("task")            # same run, offline; responses come back as
                                  # real SDK types, so resp.choices[0] works
```

## Supported providers

| Provider | Patch | Extra |
|---|---|---|
| OpenAI (chat + Responses API, incl. `.parse`/`.stream`/`with_raw_response`) | `patch_openai()` | `[openai]` |
| Anthropic (Messages + beta Messages) | `patch_anthropic()` | `[anthropic]` |
| litellm | `patch_litellm()` | `[litellm]` |
| Mistral | `patch_mistral()` | `[mistral]` |
| Cohere (v2 + legacy v1 embed/rerank) | `patch_cohere()` | `[cohere]` |
| Google Gemini (`google-genai`) | `patch_gemini()` | `[gemini]` |
| Groq | `patch_groq()` | `[groq]` |
| Together AI | `patch_together()` | `[together]` |
| Fireworks | `patch_fireworks()` | `[fireworks]` |
| Model Context Protocol | `patch_mcp()` | `[mcp]` |

Embeddings (`.embeddings.create`, `litellm.embedding`, etc.) are recorded
too, everywhere they exist. Also covered for free, no matter which SDK
constructs it: anything reached by pointing the *openai* SDK's `base_url` at
it — Azure OpenAI, DeepSeek, OpenRouter, Ollama's `/v1`, vLLM. One sharp
exception worth knowing: Groq/Together AI/Fireworks/xAI each also ship their
own dedicated Python package, so agent code built directly on those needs
the matching native patcher above, not `patch_openai()`. Full detail,
per-provider caveats, and tested SDK version ranges: **[docs/providers.md](docs/providers.md)**.

## Use cases

### 🔁 Regression-test your agent in CI, for free

```python
@pytest.mark.rewind_trace("traces/flight.rewind")
def test_flight(rewind_session):
    result = run_agent("find me a flight")
    assert "SFO" in result["final"]
```

Plain `pytest` replays the trace offline — no API key needed in CI, ever.
`pytest --rewind-record` runs the marked tests live and (re-)records their
golden traces, when your agent's behavior is supposed to change: the
cassette workflow in one flag.

### 🐛 Reproduce a bug without the API

```python
with rewind.record("bug-42.rewind"):
    agent.run(customer_input)     # capture it live, once

with rewind.replay("bug-42.rewind"):
    agent.run(customer_input)     # debug it offline, as many times as you want, $0
```

### ⏱️ Time-travel debug a failing multi-step run

```python
with rewind.replay("run.rewind", break_at=5):
    agent.run(task)   # pauses inside breakpoint() at step 5 -- real stack, real state
```

Ordinary `pdb`/`ipdb` work here since replay is deterministic and in-process.
Fork from any step to test a fix without re-running the whole thing:

```python
with rewind.fork("run.rewind", at=5, save_as="fixed.rewind") as session:
    agent.run(task)    # steps 0-4 replayed from the trace, step 5+ goes live
```

### 🔍 See exactly where two runs diverged

```
$ rewind diff before.rewind after.rewind
```

Aligns two runs step-by-step and points at the *first* divergence — the
step where the model answered differently and everything downstream
followed. Exit code 1 on divergence makes it a one-line CI regression gate.

### 💰 Know what a run cost

```
$ rewind stats run.rewind
tokens & cost (estimated):
  model                 calls      input     output       cost
  gpt-4o-2024-08-06         2       2400        680    $0.0128
```

Tokens are exact (read from the recorded `usage`); the dollar figure is a
best-effort estimate you can override with `--price MODEL=IN/OUT` or hide
with `--no-cost`.

### 🧪 Prove your tests never hit a live API

```python
backend.impl = rewind.NeverCalled()   # raises if anything reaches it live
with rewind.replay("traces/flight.rewind"):
    result = agent.run("find me a flight")
```

`ScriptedLLM` (a scriptable fake model) and `NeverCalled` are exported for
deterministic agent tests without an API key at all. For CI,
`replay(path, require_full_consumption=True)` turns leftover recorded events
into a hard failure — every recorded event must actually be served for the
run to pass.

## CLI reference

```
$ rewind show run.rewind -v          # dump every step
$ rewind show run.rewind -i          # step through it: full-screen TUI on a
                                     # terminal, line stepper otherwise
$ rewind stats run.rewind            # counts by kind, errors, time, tokens + est. cost
$ rewind diff a.rewind b.rewind      # where did two runs diverge? exit 1 if they did
$ rewind compact run.rewind          # gzip a finished trace for archival
```

`rewind show --tui` opens a two-pane curses browser: event list on the left,
selected step's request/response on the right (`j`/`k` move, `/` filters,
`enter` expands JSON, `q` quits). It degrades gracefully with no terminal —
a pipe, CI, `rewind show | less` all fall back to a line stepper
automatically. Full command and flag reference: **[docs/api-reference.md](docs/api-reference.md#cli)**.

## Replay matching strategies

| mode | behavior | use for |
|---|---|---|
| `strict` | request must match the recording exactly; drift raises `DivergenceError` with a diff | CI / regression tests |
| `ordered` | serve by position, warn on payload drift | exploratory debugging after code changes |
| `fuzzy` | serve the most similar recorded request within a look-ahead window; warns on approximate matches and skipped steps | replaying old traces against refactored agents |
| `parallel` | serve by request *identity* (exact fingerprint), order-independent | agents that fire tools concurrently (`asyncio.gather`, `asyncio.to_thread`) |

```python
with rewind.replay("run.rewind", match="fuzzy"):
    ...
```

Custom matchers, the embedding-similarity backend, and the full mechanics
behind each strategy: **[docs/internals.md](docs/internals.md#replay-matching-in-more-depth)**.

## Big traces & multimodal runs

Trace size is driven by payload *bytes*, not step count. What actually blows
traces up is multimodal content re-sent every turn — solved with one kwarg:

```python
with rewind.record("run.rewind", externalize=64_000):   # bytes threshold
    agent.run("describe these screenshots")
```

`externalize=` stores any payload leaf at or above the threshold as a
content-addressed file under `run.rewind.blobs/`, so a vision agent
re-sending one 1 MB image for 200 turns writes a **72 KB** trace plus **one**
blob instead of ~200 MB — replay identity is unchanged. Lazy loading is
automatic (replay memory is O(events), not O(bytes)), and
`rewind compact run.rewind` gzips a finished trace for archival (10–50× on
repeated-history runs). Details and measurements: **[docs/internals.md](docs/internals.md#scale-characteristics)**.

## Redacting secrets

```python
def scrub(payload):                    # any callable: payload -> payload
    if isinstance(payload, dict):
        payload.pop("api_key", None)
    return payload

with rewind.record("run.rewind", redact=scrub):
    ...
```

Redaction runs before anything touches disk, and never affects replay
identity — a call's fingerprint is taken over the un-redacted request, so
scrubbing a payload can't make a trace fail to replay.

## What gets captured

rewind records the **nondeterminism boundary**, not your agent:

- **LLMs** — `@rewind.llm` wraps any provider callable, or skip the
  decorator entirely with `auto_patch()` / the provider-specific patches above.
- **Tools** — `@rewind.tool` wraps any side-effecting callable: HTTP, DBs,
  shell. `patch_mcp()` records Model Context Protocol tool calls directly.
  Exceptions are recorded and replayed too.
- **Memory** — in-process state (message history, scratchpads) needs no
  capture: it's deterministic given the same llm/tool responses. External
  memory (vector stores, Redis) *is* a nondeterminism source — wrap reads
  with `@rewind.memory` so `stats`/`diff` can separate "the model changed
  its answer" from "retrieval returned different context."

All three decorators, and everything under *Use cases* above, work on
`async def` functions unchanged, and streaming (`stream=True`) is recorded
lazily — chunks are written as your agent consumes them, so aborting a
stream early stops paying for tokens without losing the trace.

## Runnable examples

All demos run **offline against local stub servers** — no API key needed
unless you point them at a real endpoint.

| example | what it shows |
|---|---|
| `examples/time_travel_demo.py` | fork at any step, diff the cascade, watch strict matching catch a tampered replay |
| `examples/real_agent_demo.py` | the real OpenAI SDK: record, kill the server, replay offline, diff a regression |
| `examples/async_streaming_demo.py` | an async OpenAI client streaming over SSE, replayed chunk-for-chunk |
| `examples/langgraph_demo.py` | an unmodified LangGraph ReAct agent, captured with zero code changes |
| `examples/openai_agents_demo.py` | an unmodified OpenAI Agents SDK agent, recorded and replayed offline |
| `examples/smolagents_demo.py` | an unmodified smolagents agent captured via one `patch_litellm()` call |
| `examples/parallel_agents_demo.py` | three agents running concurrently, replayed with `match="parallel"` |

## Learn more

- **[docs/providers.md](docs/providers.md)** — full provider coverage detail, the `base_url` trap, compatibility & tested SDK versions
- **[docs/api-reference.md](docs/api-reference.md)** — every export, one line each
- **[docs/known-limitations.md](docs/known-limitations.md)** — honest edges, none silent
- **[docs/internals.md](docs/internals.md)** — trace format, determinism, concurrency model, matching internals
- **[CHANGELOG.md](CHANGELOG.md)** — release history

## Contributing

Issues and PRs welcome — see [CHANGELOG.md](CHANGELOG.md) for what's already
shipped, [docs/providers.md](docs/providers.md#roadmap) for provider work
that's designed but not yet built (xAI, AWS Bedrock), and per-run cost
budgets / HTML trace export as the two open feature ideas beyond that.

## License

MIT
