Metadata-Version: 2.4
Name: corrobo
Version: 0.1.3
Summary: A runtime verification layer that independently verifies AI/tool actions against real system state.
Author-email: Fahad <fahadrahiman10@gmail.com>
License: PolyForm-Noncommercial-1.0.0
Project-URL: Homepage, https://pypi.org/project/corrobo/
Keywords: ai-agents,verification,mcp,say-do-gap,guardrails,observability,postgres,attestation
Classifier: Programming Language :: Python :: 3
Classifier: License :: Other/Proprietary License
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: postgres
Requires-Dist: psycopg[binary]>=3; extra == "postgres"
Provides-Extra: mcp
Requires-Dist: mcp>=2; extra == "mcp"
Provides-Extra: attest
Requires-Dist: cryptography>=41; extra == "attest"
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"
Requires-Dist: cryptography>=41; extra == "dev"
Dynamic: license-file

# Corrobo

**When an AI agent says "done", Corrobo reads the real system and rules VERIFIED, SAY-DO GAP or UNVERIFIED.**

An agent's tool call can *report* success and still not have done the thing. Corrobo wraps the tool,
reads the **real** before/after state through its own read-only connection, and checks it against the
effect the action **promised**. A tool that lies, or silently fails, is caught; one that genuinely worked
is proven, with a signed verdict. It never trusts the tool's own response; it reads the world.

To be precise about the word: Corrobo verifies **observable effects**. It answers "did the rows this
contract names end up the way the contract says", not "did the agent behave correctly in some absolute
sense". The contract is yours; Corrobo holds the agent to it.

## See it in thirty seconds

No key, no network, nothing to configure. Two commands:

```bash
pip install corrobo
corrobo demo
```

What you will see: a receptionist agent, three callers, one booking table. For each caller it prints,
top to bottom, what your stack saw and what the caller heard:

```
3) a taken slot, the agent says it is booked anyway
   constraint:  UNIQUE(date, time)  -> held: insert refused
   tool said:   unavailable
   log said:    TOOL -> book_appointment DENIED (taken) date='2026-10-05' time='3:00 PM'
   agent said:  "You're all set! See you on October 5th at 3:00 PM."
   table says:  3:00 PM -> someone else
   CORROBO:     SAY-DO GAP  - the real state did not match the expected effect
```

The constraint held. The tool said no. The log said DENIED. The caller was still told "all set".
Every layer below the reply did its job; none of them tests the sentence. Corrobo is the line that
compares the sentence to the table. Caller 1 gets **VERIFIED** (free slot, really booked) and caller 2
**UNVERIFIED** (taken slot, the agent said so: an honest failure, not an accusation).

The replies in callers 1 and 2 are verbatim from a live run of a real agent (below); caller 3 is
scripted and labelled so on screen, because in that run the agent was honest 4 times out of 4. It is
the conversation Corrobo exists for on the day the agent is not.

To do the same to your own tool, wrap it: [Thirty seconds, no setup](#thirty-seconds-no-setup).

## Proof, not a promise: a live AI receptionist, verified

On 2026-09-24 Corrobo was wired to a production booking agent (Gemini + a live Postgres) and run over
eight real conversations. Four callers asked for a slot that was already taken.

| Caller asked for | What the agent told the caller | What the database said | Verdict |
| --- | --- | --- | --- |
| 9:00 AM, already taken | "it looks like 9:00 AM is already taken" | no row for this caller | **UNVERIFIED** — honest failure, no accusation |
| 9:30 AM, free | "your cleaning is all set for October 5th at 9:30 AM" | a row for this caller, this slot | **VERIFIED** |
| taken slot; tool refused; caller told "booked" anyway | "You're all set!" | no row | **SAY-DO GAP** |

Verdicts matched the table in 8 of 8 conversations. It also caught its author twice on the way: a contract
that said "done = *any* booking on this slot" was ruled `VERIFIED_VACUOUS` because the effect already held
before the action, and a claim-reader that counted "that slot has *just been booked*" as a booking claim was
exposed by printing its disagreements. A verifier that flags a lazy contract is the point.

Measured before that on public agent benchmarks (τ-bench retail, AgentDojo, Agent-Diff, Box): on the Box
suite **59% of the agent's success claims were false**, and neither the tool's return value nor a
transcript-reading judge caught them. Reading the database did.

## Thirty seconds, no setup

```python
import sqlite3
from corrobo import Corrobo, SQLiteStateSource

con = sqlite3.connect("demo.db")
con.execute("CREATE TABLE IF NOT EXISTS bookings (id INTEGER PRIMARY KEY, slot TEXT UNIQUE, who TEXT)")
con.commit()

def book(slot, who):                      # a tool that LIES: says ok even when the slot was taken
    try:
        con.execute("INSERT INTO bookings (slot, who) VALUES (?, ?)", (slot, who)); con.commit()
    except sqlite3.IntegrityError:
        pass
    return {"ok": True, "message": f"You're all set for {slot}!"}

vd = Corrobo(mode="monitor", state=SQLiteStateSource(path="demo.db"))

for who in ("Alex", "Jordan"):            # second caller gets the same slot
    vd.check("book", {"slot": "Tue 3pm", "who": who}, lambda: book("Tue 3pm", who),
             succeeded=lambda r: r["ok"],
             reads=lambda a: [("bookings", {"slot": a["slot"]})],
             effect=lambda before, after, a: any(r["who"] == a["who"] for r in after["bookings"].values()))
    print(who, "->", vd.last()["status"])

# Alex -> VERIFIED
# Jordan -> SAY-DO GAP        (told "all set"; the table says otherwise)
```

## Start here: the six names you need

The package exports a lot (mining, invariants, attestations, the MCP proxy). Ignore all of it until
you need it. Day one is:

| Name | What it is |
| --- | --- |
| `Corrobo(mode, state)` | one instance per app; `"monitor"` observes, `"enforce"` blocks and raises |
| `SQLiteStateSource` / `PostgresStateSource` / `RestStateSource` | where the truth lives; read-only |
| `@vd.verify(reads=..., effect=...)` | decorate your own function |
| `vd.check(name, args, thunk, reads=..., effect=...)` | the same, for a tool you only have a handle to |
| `vd.last()` | the verdict: `["status"]`, `.explain()`, `.get("reason")` |
| `mine_tool(vd, name, fn, arg_list)` | let Corrobo draft the contract by watching the tool (below) |

`help(Corrobo)` and `help(Corrobo.check)` describe every argument.

## Do not want to write the contract? Let it be proposed

Point `mine_tool` at your function with a handful of calls. It snapshots before and after each one,
finds what always changes and how it relates to the arguments, mutation-tests each rule, and returns
**proposals**: things a human accepts or rejects, never contracts that apply themselves.

```python
from corrobo import Corrobo, SQLiteStateSource, mine_tool, render_proposals

vd = Corrobo(mode="monitor", state=SQLiteStateSource(path="app.db"))

def upgrade(user_id, plan):
    con.execute("UPDATE users SET plan=? WHERE id=?", (plan, user_id)); con.commit()

props = mine_tool(vd, "upgrade", lambda a: upgrade(**a),
                  [{"user_id": i, "plan": "pro"} for i in range(1, 7)],
                  reads=lambda a: [("users", {"id": a["user_id"]})])
print(render_proposals(props))
# [1] `upgrade` always set `plan` to whatever was passed as `plan` (6 calls) ...   score 18/18 mutations caught
# [2] `upgrade` changed exactly 1 row(s) in all 6 of 6 calls. Should more than 1 be refused?
# [3] `upgrade` only ever touched ['users']. Should touching anything else be refused?

# accept one: an effect proposal is a contract in one line
rule = next(p for p in props if p.kind == "effect").as_effect()
vd.check("upgrade", {"user_id": 7, "plan": "pro"}, lambda: upgrade(7, "pro"),
         reads=lambda a: [("users", {"id": a["user_id"]})], effect=rule)
print(vd.last()["status"])   # VERIFIED
```

It really runs the tool six times, so use a copy of the data. Fewer than five calls proposes nothing,
on purpose: a rule learned from three calls is a guess wearing a badge. Shape proposals (row counts,
collections touched) become invariants through `accept(p)`.

```python
from corrobo import Corrobo, RestStateSource

vd = Corrobo(mode="monitor", state=RestStateSource(
    "https://api.github.com",
    resources={"issue": {"path": "/repos/OWNER/REPO/issues/{number}", "pk": "number"}},
    headers={"Authorization": f"Bearer {TOKEN}", "Accept": "application/vnd.github+json"}))

vd.check("rename_issue", {"number": 1, "new_title": "Fixed"},
         lambda: github_patch(1, {"title": "Fixed"}),      # your real tool call
         succeeded=lambda r: r.ok,                           # HTTP 2xx = the tool claims success
         reads=lambda a: [("issue", {"number": a["number"]})],
         effect=lambda before, after, a: after["issue"][a["number"]]["title"] == a["new_title"])

print(vd.last().status)        # VERIFIED  |  SAY-DO GAP  |  UNVERIFIED
print(vd.last().explain())     # a human, secrets-free explanation:
#   Action:    rename_issue  (args: new_title, number)
#   Expected:  the declared effect to hold on issue[number] (read independently, before & after)
#   Observed:  real state changed [issue #1: comments, updated_at], but the promised effect did NOT hold
#   Status:    SAY-DO GAP — the real state did not match the expected effect (say-do gap)
```

## The full stack — from "did it happen?" to "the agent can't fake success"

The say-do-gap check above is layer 1. The proven layers on top:

1. **Independent verification** — read real state; decide VERIFIED / SAY-DO GAP / UNVERIFIED (above).
2. **Cross-system outcomes** — one contract can require an effect across *multiple* systems of record
   (`MultiStateSource` + identity correlation). The whole invariant must hold, or it's a gap.
3. **Independent verifier, out of process** — run the engine as its OWN process with its OWN read-only
   credentials, contracts, and signing key. The agent sends only `{contract_id, args}` — it cannot supply the
   state source, the contract, or the key (`VerifierService` / `VerifierClient`).
4. **Enforcement gate** — an authoritative success is released only on an independent VERIFIED (`OutcomeGate`).
5. **Signed success attestations** — on a genuine VERIFIED the verifier mints a short-lived **Ed25519**
   attestation bound to the operation; any consumer validates it with the **public key** (no secret). It can't
   be forged, modified, replayed, retargeted, or outlived (`OutcomeGate.accept`, `verify_attestation`).
6. **Consumption-time re-verification** — optionally re-check the effect at the moment of release to shrink the
   TOCTOU window (`accept(..., reverify=True)`).

Validated end to end on **real GitHub REST, PostgreSQL, Jira Cloud, MCP (stdio), and a live receptionist
agent (Gemini + Postgres)**, plus adversarial red-teams of the verifier trust boundary and the attestation.

## Install

```bash
pip install corrobo                 # core — pure standard library, no dependencies
pip install "corrobo[postgres]"     # + PostgresStateSource   (psycopg 3)
pip install "corrobo[mcp]"          # + drive a real MCP server over stdio (mcp)
pip install "corrobo[attest]"       # + mint/verify Ed25519 success attestations (cryptography)
```

```python
from corrobo import Corrobo, CorroboSayDoGap, CorroboUncertain, CorroboBlocked
```

Python 3.9+. The core imports nothing outside the standard library; the extras are imported lazily, so
installing the core never pulls a driver you do not use.

## How it works — two things you declare

| You declare | What it is |
|---|---|
| `reads(args)` | the **footprint** — the exact rows the action touches, e.g. `[("issue", {"number": 1})]`. Corrobo reads **only** these, before and after. Nothing else is read, diffed, or logged. |
| `effect(before, after, args)` | the **promise** — a boolean over the real before/after state, e.g. `after["issue"][1]["title"] == "Fixed"`. |

Corrobo snapshots the footprint, runs the tool, snapshots again, and evaluates the effect against **real
state** — not the tool's return value.

## The verdicts

| Status | Meaning |
|---|---|
| **VERIFIED** | the promised effect is actually present in real state |
| **SAY-DO GAP** | the tool **claimed success** but real state does not match the promise (it lied, or silently failed) |
| **UNVERIFIED** | the result could not be established — the tool **reported failure** (an honest failure, not a gap), the state couldn't be read, or a **concurrent modification** made the outcome unattributable |
| **BLOCKED** | a pre-check (policy/precondition) refused the action before it ran (enforce mode) |

An honest tool failure is distinguished from a lie: pass `succeeded=lambda r: ...` (e.g. an HTTP 2xx check,
or an MCP `is_error` flag) and a tool that reports failure becomes **UNVERIFIED (action-failed)**, never a
say-do gap.

## Monitor vs enforce

- **`mode="monitor"`** (safe default) — run the tool, verify, record evidence, **never interfere**.
- **`mode="enforce"`** — a pre-check can **block** the action; a say-do gap **raises `CorroboSayDoGap`**
  (stop downstream); an unestablished result raises **`CorroboUncertain`** (fail-safe). An honest tool
  failure does **not** raise (its own failure result is returned).

## Adapters (StateSource) — one engine, any backend

The engine reads state through a `StateSource`; only the adapter changes.

```python
from corrobo import RestStateSource        # any REST/HTTP API (scoped GETs, Retry-After, retries)
from corrobo import PostgresStateSource     # real Postgres (consistent READ ONLY snapshot, PK auto-detect)
from corrobo import SQLiteStateSource       # a SQLite file or connection
from corrobo import CorroboMCPProxy         # sit in front of an existing MCP server; verify each tool call
```

Corrobo has been validated end-to-end against **real GitHub REST**, **real PostgreSQL**, and the **real MCP
SDK over stdio** — the same core engine, no per-system verification logic.

## Evidence

Every verification appends to a tamper-evident, hash-chained log. It stores **only hashes, the argument
keys, the scoped footprint (column names), and which columns changed** — never raw values, never secrets,
never the connection string.

```python
vd = Corrobo(mode="monitor", state=..., evidence_path="corrobo.jsonl")
vd.evidence_log.verify()     # True — the chain is intact
```

## Current guarantees (validated on real systems)

- VERIFIED is emitted only when the promised effect is actually present — decided by an **independent
  read**, never the tool's response body. Multi-condition effects and idempotent re-application verify
  correctly.
- Real say-do gaps are caught: wrong record, wrong field, partial effect, or a lying success response.
- Honest tool failures (401/403/404/HTTP errors, DB errors, tool-reported failure) → **action-failed /
  UNVERIFIED**, never a say-do gap.
- **Fail-safe under unavailability**: a read 404, a timeout, a dropped connection, or a resource that
  vanished → **UNVERIFIED, never a false VERIFIED**.
- **Concurrency**: a concurrent change to the **effect-relevant** state → UNVERIFIED / CorroboUncertain;
  **unrelated** concurrent churn (an extra comment, a bumped `updated_at`/`LastModifiedDate`, an unrelated
  row/column) does **not** cause a false failure.
- Direct-record reads are treated as strongly consistent; genuinely lagging reads are handled by a bounded
  settle window (`settle_retries`); rate limits honor a bounded `Retry-After`.

## Limitations (honest)

- **Concurrency detection is verdict-based, not field-traced**: a concurrent write that changes a footprint
  row but does **not** flip the effect's verdict (e.g. a loose/inequality effect) is not flagged. Exact-value
  effects — the common case — have no blind spot.
- A read **404 is ambiguous** (truly absent vs. no access), so a genuinely-absent target is reported as
  **UNVERIFIED (fail-safe)**, not a definitive say-do gap.
- Corrobo is a **verifier, not a lock manager or a rollback engine**: it detects and (in enforce) blocks;
  it does not undo the tool's transaction.
- Postgres contracts must declare a `reads` footprint (whole-DB reads are refused for production safety),
  and every table read needs a primary key.
- Expiring-token refresh for REST auth is not built in yet (static headers today).

## What ships in the package

`corrobo/` only: the engine, the REST / Postgres / SQLite adapters, the MCP proxy, contract mining,
invariants, frame conditions, drift detection, the retry and ambiguity gates, the out-of-process verifier,
the enforcement gate and Ed25519 attestations. About 7,700 lines, with roughly 11,000 lines of tests behind
them (83 files: engine tests plus live validation campaigns against GitHub, Postgres, Jira and MCP).

## Status

Beta. The engine has had three adversarial review passes, a full CodeRabbit sweep, a public-benchmark
campaign and one live production workload. The verdict logic is stable; what is still moving is
deployment hardening (key distribution, a shared replay store, TLS, multi-instance) and the ergonomics
people ask for. **If you run an agent that writes to a real system, the most useful thing you can do is run
Corrobo in monitor mode beside it for a week and tell the author what it found.**

## Feedback

Bugs, false verdicts, and "it should have caught this" reports are the most valuable thing you can send.
Email **fahadrahiman10@gmail.com** and include the `explain()` output and the evidence entry (both are
secrets-free by construction). Commercial licensing goes to the same address.

## License

PolyForm Noncommercial 1.0.0. Free to use, copy, modify and share for noncommercial purposes: personal
projects, research, teaching, and noncommercial organizations. **Commercial use needs a separate license:**
contact the author. See `LICENSE` for the full terms.
