Metadata-Version: 2.5
Name: fannypack-agents
Version: 0.2.0
Summary: Fast, reversible tool calls for LLM agents. Your tools, strapped on, ready before you reach for them.
Project-URL: Homepage, https://github.com/nikhilkulkarni1755/fannypack-agents
Project-URL: Repository, https://github.com/nikhilkulkarni1755/fannypack-agents
Project-URL: Issues, https://github.com/nikhilkulkarni1755/fannypack-agents/issues
Author: Nikhil Kulkarni
License: Apache-2.0
License-File: LICENSE
Keywords: agents,compensation,latency,llm,mcp,tool-calling,undo
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.10
Requires-Dist: httpx>=0.27
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Provides-Extra: http2
Requires-Dist: h2>=4; extra == 'http2'
Provides-Extra: mcp
Requires-Dist: mcp>=1.0; extra == 'mcp'
Description-Content-Type: text/markdown

# fannypack

**Fast, reversible tool calls for LLM agents.** Your tools strapped on and within
reach: the runtime already has the likely call in flight before the model
finishes asking for it, and every call it makes is recorded well enough to be
taken back afterwards.

Two claims, both measured:

1. **Latency shaving** — 1.2–1.8x off a tool-calling turn, taken from the parts
   of the latency that are the runtime's fault rather than the model's. Bounded
   by tool latency: with tools under ~50ms there is nothing to shave (a coding
   belt measured 1.0–1.17x); the gain grows with slower, uneven, or repeated
   calls.
2. **Selective undo** — take back one action and leave every action that never
   depended on it standing. The runtime says which of three things an action is
   (nothing to undo, reversible, irreversible) and refuses rather than pretends.

## Install

```bash
pip install fannypack-agents        # the bare name on PyPI is an unrelated package; the import is `fannypack`
```

or from source:

```bash
git clone https://github.com/nikhilkulkarni1755/fannypack-agents && cd fannypack-agents
pip install -e .                    # one dependency: httpx
```

Python 3.10+. Provider keys go in `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, or
`FIREWORKS_API_KEY`; Ollama needs no key.

## Quickstart: a bank agent

Register tools with a decorator. Each one says what it does to the world, and
the ones that can be taken back say how.

```python
import asyncio
from fannypack import Agent, Effect, Pack, providers

pack = Pack()
balances = {"acct_alice_0001": 1000.0, "acct_bob_0002": 250.0}
transfers = []

@pack.tool(effect=Effect.READ_ONLY)                      # a read: cacheable, safe to run early
async def get_balance(account_id: str) -> dict:
    """Balance of one account."""
    return {"account_id": account_id, "balance": balances[account_id]}

@pack.tool(effect=Effect.IDEMPOTENT_WRITE)               # a write; the same reference twice moves money once
async def transfer(from_account: str, to_account: str, amount: float, reference: str) -> dict:
    """Move money between accounts."""
    balances[from_account] -= amount
    balances[to_account] += amount
    transfers.append(reference)
    return {"transfer_id": f"txn_{len(transfers):06d}", "from_account": from_account,
            "to_account": to_account, "amount": amount}

@pack.compensator("transfer")                            # this is what makes transfer reversible
async def reverse_transfer(result: dict) -> dict:
    balances[result["to_account"]] -= result["amount"]
    balances[result["from_account"]] += result["amount"]
    return {"reversed": result["transfer_id"]}

@pack.tool(effect=Effect.NON_IDEMPOTENT_WRITE)           # no compensator: the notes are gone
async def withdraw_cash(account_id: str, amount: float) -> dict:
    """Dispense cash at an ATM."""
    balances[account_id] -= amount
    return {"dispensed": amount}

async def main():
    agent = Agent(pack, providers.from_spec("anthropic:claude-sonnet-5"))
    run = await agent.run("Move 100 from acct_alice_0001 to acct_bob_0002 with reference 'rent-sep', "
                          "then confirm both balances.")
    print(run.answer)
    print(run.metrics.as_dict())

asyncio.run(main())
```

Schemas come from your type hints; `Annotated[str, "..."]` adds a parameter
description. No base classes, no rewrite of your functions.

### Selective undo

Suppose the transfer was a mistake.

```python
txn = next(a for a in run.actions() if a.tool == "transfer")

print(run.plan_undo(txn.id).explain())
# undo a_7f2c19 in run_4b81:
#   compensate a_7f2c19 (transfer) [target] -- compensating via reverse_transfer

await run.undo(txn.id)                      # the money is back; the balance reads that
                                            # followed are untouched -- they never depended on it
```

And suppose the model had withdrawn cash instead:

```python
cash = next(a for a in run.actions() if a.tool == "withdraw_cash")
await run.undo(cash.id)
# Irreversible: withdraw_cash registers no compensation
```

There is no `force` flag. Every action resolves to one of three states —
nothing to undo (a read), reversible (a write with a compensator), irreversible
(a write without one) — and the plan tells you which before anything runs.

A write whose inverse needs the *old* state — an edit, an overwrite — gets it
from a snapshot the runtime takes just before the write and stores on the
action:

```python
@pack.snapshotter("write_file")
def before_write(args: dict) -> dict:
    return {"content": files.get(args["path"])}

@pack.compensator("write_file")
def restore(args: dict, before: dict) -> dict:
    files[args["path"]] = before["content"]
    return {"restored": args["path"]}
```

What makes this *selective* is the ledger: it records which later calls used
which earlier results (a `transfer_id` returned by one call and passed to
another), so undoing one action cascades only through the calls that actually
consumed it. `run.why(action_id)` shows that chain; `run.verify()` confirms the
history was appended to, never rewritten.

### Restart from step N

```python
resumed = await agent.resume(run.id, from_action=some_model_turn.id)
```

The transcript up to that point is rebuilt from the ledger, and every call the
original run completed is served back from the record instead of executed
again — reads within their freshness window, writes unconditionally. Twelve of
twelve resumed bank transfers across four real models moved the money exactly
once.

## The vocabulary

| Term | What it means | Where |
|---|---|---|
| **Pack** | Your tools, registered once, each with an effect class | `Pack`, `Effect` |
| **Ledger** | The append-only, hash-chained record of what happened and what depended on what | `Ledger` |
| **Latency shaving** | Taking off every part of a turn's time that is the runtime's to take: the flags in the next table | `Options` |
| **Selective undo** | Take one action back; independent actions stand; irreversible ones refuse | `run.undo` |
| **Reach-ahead** | Start a *guessed* read-only call before the model asks for it | `Options.speculate` |
| **Pre-dispatch** | Start the calls a request of this shape *always* needs, from a learned table: deterministic, one lookup, the same schedule every time | `Options.policy` |
| **One reach** | Several dependent calls in a single model turn — `"$1.next"` feeds step 1 into step 2; each step starts the moment its JSON closes, while the rest of the plan is still streaming | `Options.plans` |
| **Pre-check** | Does this string need a tool, and which? ~60µs, deterministic | `Classifier` |

Latency shaving is these flags, each measurable alone:

| Flag | Removes |
|---|---|
| `routing` | The tokens of every schema you *don't* need this turn (65–85% of input) |
| `stream_ahead` | The gap between "the model decided" and "the call started" |
| `parallel` | Queue time behind independent calls |
| `caching` | Repeated and duplicated I/O |
| `speculate` / `policy` | The first call's latency entirely |
| `plans` | N−1 of the N model round-trips in a dependent chain |

`Options.baseline()` turns everything off — that is how most agent loops run
today, and what the numbers below compare against. It is an honest baseline:
measured equal to a hand-written sequential SDK loop to within 10ms on every
shape tested.

Which flags matter for which shape, measured against peer runtimes (OpenAI
Agents SDK, Pydantic AI, LangGraph) on a scripted model:

- **Repeated reads** — the cache is the largest default-on win, 1.27x at 0.5s
  tools rising to 1.59x at 2s. No peer runtime has one.
- **Uneven fan-out** (one slow call among fast ones) — stream-ahead wins by
  1.05–1.11x; on a *homogeneous* fan-out every runtime that runs calls
  concurrently ties within 20ms, and all three peers do.
- **Dependent chains** — only `plans` help, and a plan is worth about one model
  round-trip per step saved, so its value is set by the model's per-turn time:
  ~2s on Sonnet, less on faster models. Steps now run while the plan is still
  streaming (0.4–0.9s of tool time under decode on gpt-oss chains); on a
  scripted model that is a 5-step chain in 1.15s instead of 1.75s.
- **Small belts** — `routing` is inert until the pack is larger than `route_k`
  (12); it exists for the 50-tool case.

## Findings

Three rounds, four models (`claude-sonnet-5`, `claude-haiku-4-5`, `gpt-5.1`,
`gpt-oss-120b`), a 47–50 tool belt, every run correctness-gated so a
configuration cannot get faster by doing less. Full tables in
[`docs/results.md`](docs/results.md), [`docs/results-2.md`](docs/results-2.md)
and [`docs/results-3.md`](docs/results-3.md); raw samples and ledgers in
[`bench-results/`](bench-results/).

**Claim 1 — latency shaving, 1.2–1.8x.** On independent-call workloads the runtime removes its
share: gpt-5.1 `fan_out` 13.0s → 7.6s, Sonnet 18.0s → 14.4s, the clearest single
mechanism being parallel fan-out taking `queued` from 12.7s to 0. On strictly
dependent chains the ordinary mechanisms are neutral — nothing to overlap — and
*one reach* is what moves them: gpt-5.1 `chain_8` 13.7s → 7.6s, nine model
turns to two or three. Pre-dispatch takes a warm `research_chain_3` from 11.3s
to 5.8s on gpt-oss at 100% precision.

**Claim 2 — selective undo.** In the suite, undoing a transfer reverses exactly
the transfer; undoing a route change restores the route and leaves the brake
applied; undoing a cash withdrawal is refused with the reason. Restart from any
model turn replays the record: 12/12 exactly-once.

**What does not help, measured.** `compact_state` loses on most workloads.
Reach-ahead from the goal text wasted about one read per run and is off by
default. Routing *hurt* on Sonnet chains until the routed prefix was frozen and
large enough to prompt-cache. Haiku never uses one-reach plans at all. The
runtime's own time is 2–3ms a turn, under 8ms at max; the model is where the
seconds are.

**Do models need fine-tuning to call tools faster?** No, for round-trips —
plans and pre-dispatch get that from frontier models by prompting. What a
fine-tune *would* buy, read off the ledgers with `fannypack mine`: gpt-oss
emits sequential-but-independent calls 48% of the time (plans already fix
that), Haiku writes ~15 tokens of prose before each call, gpt-5.1 and Sonnet
nothing. [`docs/research.md`](docs/research.md) has the literature.

## Run the experiments

```bash
fannypack bench --provider anthropic:claude-sonnet-5 --repeats 3 --json out.json
fannypack report out.json
fannypack compare bench-results/*.json

cd suite && uv sync            # the external suite: a bank, a car, a camera, a research chain
uv run fannypack-suite chain  --provider openai:gpt-5.1 --db /tmp/ledgers
uv run fannypack-suite policy --provider fireworks:accounts/fireworks/models/gpt-oss-120b
uv run fannypack-suite mine     /tmp/ledgers/*.db
uv run fannypack-suite classify /tmp/ledgers/*.db
```

## Elsewhere

- **As a library**: `PackServer` gives an agent loop you already have the
  ledger, the cache and selective undo without adopting `Agent`; `Router`,
  `ResultCache`, `Ledger` and `plan_undo` work on their own.
- **Over MCP** (`pip install -e '.[mcp]'`): `examples/serve_mcp.py` serves a
  pack to Claude or ChatGPT with `fannypack_undo` and `fannypack_why` as tools.
- **CLI**: `fannypack inspect | why | graph | verify | undo | policy | mine | classify`.
- **Not this**: a tracing dashboard, a durable execution engine, an agent graph
  framework, a replacement for MCP, a sandbox. [`docs/design.md`](docs/design.md)
  has the reasoning and the known limits.

If you find it useful, a star on the repo helps other people find it.

Apache-2.0.
