Metadata-Version: 2.4
Name: corrobo
Version: 0.1.0
Summary: A runtime verification layer that independently verifies AI/tool actions against real system state.
Author: Fahad
License: PolyForm-Noncommercial-1.0.0
Keywords: ai-agents,verification,mcp,say-do-gap,guardrails,observability,postgres,attestation
Classifier: Programming Language :: Python :: 3
Classifier: License :: Other/Proprietary License
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: postgres
Requires-Dist: psycopg[binary]>=3; extra == "postgres"
Provides-Extra: mcp
Requires-Dist: mcp>=2; extra == "mcp"
Provides-Extra: attest
Requires-Dist: cryptography>=41; extra == "attest"
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"
Requires-Dist: cryptography>=41; extra == "dev"
Dynamic: license-file

# Corrobo

**When an AI agent says "done", Corrobo reads the real system and rules VERIFIED, SAY-DO GAP or UNVERIFIED.**

An agent's tool call can *report* success and still not have done the thing. Corrobo wraps the tool,
reads the **real** before/after state through its own read-only connection, and checks it against the
effect the action **promised**. A tool that lies, or silently fails, is caught; one that genuinely worked
is proven, with a signed verdict. It never trusts the tool's own response; it reads the world.

```bash
pip install corrobo
```

## Proof, not a promise: a live AI receptionist, verified

On 2026-09-24 Corrobo was wired to a production booking agent (Gemini + a live Postgres) and run over
eight real conversations. Four callers asked for a slot that was already taken.

| Caller asked for | What the agent told the caller | What the database said | Verdict |
| --- | --- | --- | --- |
| 9:00 AM, already taken | "it looks like 9:00 AM is already taken" | no row for this caller | **UNVERIFIED** — honest failure, no accusation |
| 9:30 AM, free | "your cleaning is all set for October 5th at 9:30 AM" | a row for this caller, this slot | **VERIFIED** |
| taken slot; tool refused; caller told "booked" anyway | "You're all set!" | no row | **SAY-DO GAP** |

Verdicts matched the table in 8 of 8 conversations. It also caught its author twice on the way: a contract
that said "done = *any* booking on this slot" was ruled `VERIFIED_VACUOUS` because the effect already held
before the action, and a claim-reader that counted "that slot has *just been booked*" as a booking claim was
exposed by printing its disagreements. A verifier that flags a lazy contract is the point.

Measured before that on public agent benchmarks (τ-bench retail, AgentDojo, Agent-Diff, Box): on the Box
suite **59% of the agent's success claims were false**, and neither the tool's return value nor a
transcript-reading judge caught them. Reading the database did.

## Thirty seconds, no setup

```python
import sqlite3
from corrobo import Corrobo, SQLiteStateSource

con = sqlite3.connect("demo.db")
con.execute("CREATE TABLE IF NOT EXISTS bookings (id INTEGER PRIMARY KEY, slot TEXT UNIQUE, who TEXT)")
con.commit()

def book(slot, who):                      # a tool that LIES: says ok even when the slot was taken
    try:
        con.execute("INSERT INTO bookings (slot, who) VALUES (?, ?)", (slot, who)); con.commit()
    except sqlite3.IntegrityError:
        pass
    return {"ok": True, "message": f"You're all set for {slot}!"}

vd = Corrobo(mode="monitor", state=SQLiteStateSource(path="demo.db"))

for who in ("Alex", "Jordan"):            # second caller gets the same slot
    vd.check("book", {"slot": "Tue 3pm", "who": who}, lambda: book("Tue 3pm", who),
             succeeded=lambda r: r["ok"],
             reads=lambda a: [("bookings", {"slot": a["slot"]})],
             effect=lambda before, after, a: any(r["who"] == a["who"] for r in after["bookings"].values()))
    print(who, "->", vd.last()["status"])

# Alex -> VERIFIED
# Jordan -> SAY-DO GAP        (told "all set"; the table says otherwise)
```

```python
from corrobo import Corrobo, RestStateSource

vd = Corrobo(mode="monitor", state=RestStateSource(
    "https://api.github.com",
    resources={"issue": {"path": "/repos/OWNER/REPO/issues/{number}", "pk": "number"}},
    headers={"Authorization": f"Bearer {TOKEN}", "Accept": "application/vnd.github+json"}))

vd.check("rename_issue", {"number": 1, "new_title": "Fixed"},
         lambda: github_patch(1, {"title": "Fixed"}),      # your real tool call
         succeeded=lambda r: r.ok,                           # HTTP 2xx = the tool claims success
         reads=lambda a: [("issue", {"number": a["number"]})],
         effect=lambda before, after, a: after["issue"][a["number"]]["title"] == a["new_title"])

print(vd.last().status)        # VERIFIED  |  SAY-DO GAP  |  UNVERIFIED
print(vd.last().explain())     # a human, secrets-free explanation:
#   Action:    rename_issue  (args: new_title, number)
#   Expected:  the declared effect to hold on issue[number] (read independently, before & after)
#   Observed:  real state changed [issue #1: comments, updated_at], but the promised effect did NOT hold
#   Status:    SAY-DO GAP — the real state did not match the expected effect (say-do gap)
```

## The full stack — from "did it happen?" to "the agent can't fake success"

The say-do-gap check above is layer 1. The proven layers on top:

1. **Independent verification** — read real state; decide VERIFIED / SAY-DO GAP / UNVERIFIED (above).
2. **Cross-system outcomes** — one contract can require an effect across *multiple* systems of record
   (`MultiStateSource` + identity correlation). The whole invariant must hold, or it's a gap.
3. **Independent verifier, out of process** — run the engine as its OWN process with its OWN read-only
   credentials, contracts, and signing key. The agent sends only `{contract_id, args}` — it cannot supply the
   state source, the contract, or the key (`VerifierService` / `VerifierClient`).
4. **Enforcement gate** — an authoritative success is released only on an independent VERIFIED (`OutcomeGate`).
5. **Signed success attestations** — on a genuine VERIFIED the verifier mints a short-lived **Ed25519**
   attestation bound to the operation; any consumer validates it with the **public key** (no secret). It can't
   be forged, modified, replayed, retargeted, or outlived (`OutcomeGate.accept`, `verify_attestation`).
6. **Consumption-time re-verification** — optionally re-check the effect at the moment of release to shrink the
   TOCTOU window (`accept(..., reverify=True)`).

Validated end to end on **real GitHub REST, PostgreSQL, Jira Cloud, MCP (stdio), and a live receptionist
agent (Gemini + Postgres)**, plus adversarial red-teams of the verifier trust boundary and the attestation.

## Install

```bash
pip install corrobo                 # core — pure standard library, no dependencies
pip install "corrobo[postgres]"     # + PostgresStateSource   (psycopg 3)
pip install "corrobo[mcp]"          # + drive a real MCP server over stdio (mcp)
pip install "corrobo[attest]"       # + mint/verify Ed25519 success attestations (cryptography)
```

```python
from corrobo import Corrobo, CorroboSayDoGap, CorroboUncertain, CorroboBlocked
```

Python 3.9+. The core imports nothing outside the standard library; the extras are imported lazily, so
installing the core never pulls a driver you do not use.

## How it works — two things you declare

| You declare | What it is |
|---|---|
| `reads(args)` | the **footprint** — the exact rows the action touches, e.g. `[("issue", {"number": 1})]`. Corrobo reads **only** these, before and after. Nothing else is read, diffed, or logged. |
| `effect(before, after, args)` | the **promise** — a boolean over the real before/after state, e.g. `after["issue"][1]["title"] == "Fixed"`. |

Corrobo snapshots the footprint, runs the tool, snapshots again, and evaluates the effect against **real
state** — not the tool's return value.

## The verdicts

| Status | Meaning |
|---|---|
| **VERIFIED** | the promised effect is actually present in real state |
| **SAY-DO GAP** | the tool **claimed success** but real state does not match the promise (it lied, or silently failed) |
| **UNVERIFIED** | the result could not be established — the tool **reported failure** (an honest failure, not a gap), the state couldn't be read, or a **concurrent modification** made the outcome unattributable |
| **BLOCKED** | a pre-check (policy/precondition) refused the action before it ran (enforce mode) |

An honest tool failure is distinguished from a lie: pass `succeeded=lambda r: ...` (e.g. an HTTP 2xx check,
or an MCP `is_error` flag) and a tool that reports failure becomes **UNVERIFIED (action-failed)**, never a
say-do gap.

## Monitor vs enforce

- **`mode="monitor"`** (safe default) — run the tool, verify, record evidence, **never interfere**.
- **`mode="enforce"`** — a pre-check can **block** the action; a say-do gap **raises `CorroboSayDoGap`**
  (stop downstream); an unestablished result raises **`CorroboUncertain`** (fail-safe). An honest tool
  failure does **not** raise (its own failure result is returned).

## Adapters (StateSource) — one engine, any backend

The engine reads state through a `StateSource`; only the adapter changes.

```python
from corrobo import RestStateSource        # any REST/HTTP API (scoped GETs, Retry-After, retries)
from corrobo import PostgresStateSource     # real Postgres (consistent READ ONLY snapshot, PK auto-detect)
from corrobo import SQLiteStateSource       # a SQLite file or connection
from corrobo import CorroboMCPProxy         # sit in front of an existing MCP server; verify each tool call
```

Corrobo has been validated end-to-end against **real GitHub REST**, **real PostgreSQL**, and the **real MCP
SDK over stdio** — the same core engine, no per-system verification logic.

## Evidence

Every verification appends to a tamper-evident, hash-chained log. It stores **only hashes, the argument
keys, the scoped footprint (column names), and which columns changed** — never raw values, never secrets,
never the connection string.

```python
vd = Corrobo(mode="monitor", state=..., evidence_path="corrobo.jsonl")
vd.evidence_log.verify()     # True — the chain is intact
```

## Current guarantees (validated on real systems)

- VERIFIED is emitted only when the promised effect is actually present — decided by an **independent
  read**, never the tool's response body. Multi-condition effects and idempotent re-application verify
  correctly.
- Real say-do gaps are caught: wrong record, wrong field, partial effect, or a lying success response.
- Honest tool failures (401/403/404/HTTP errors, DB errors, tool-reported failure) → **action-failed /
  UNVERIFIED**, never a say-do gap.
- **Fail-safe under unavailability**: a read 404, a timeout, a dropped connection, or a resource that
  vanished → **UNVERIFIED, never a false VERIFIED**.
- **Concurrency**: a concurrent change to the **effect-relevant** state → UNVERIFIED / CorroboUncertain;
  **unrelated** concurrent churn (an extra comment, a bumped `updated_at`/`LastModifiedDate`, an unrelated
  row/column) does **not** cause a false failure.
- Direct-record reads are treated as strongly consistent; genuinely lagging reads are handled by a bounded
  settle window (`settle_retries`); rate limits honor a bounded `Retry-After`.

## Limitations (honest)

- **Concurrency detection is verdict-based, not field-traced**: a concurrent write that changes a footprint
  row but does **not** flip the effect's verdict (e.g. a loose/inequality effect) is not flagged. Exact-value
  effects — the common case — have no blind spot.
- A read **404 is ambiguous** (truly absent vs. no access), so a genuinely-absent target is reported as
  **UNVERIFIED (fail-safe)**, not a definitive say-do gap.
- Corrobo is a **verifier, not a lock manager or a rollback engine**: it detects and (in enforce) blocks;
  it does not undo the tool's transaction.
- Postgres contracts must declare a `reads` footprint (whole-DB reads are refused for production safety),
  and every table read needs a primary key.
- Expiring-token refresh for REST auth is not built in yet (static headers today).

## What ships in the package

`corrobo/` only: the engine, the REST / Postgres / SQLite adapters, the MCP proxy, contract mining,
invariants, frame conditions, drift detection, the retry and ambiguity gates, the out-of-process verifier,
the enforcement gate and Ed25519 attestations. About 7,700 lines, with roughly 11,000 lines of tests behind
them (83 files: engine tests plus live validation campaigns against GitHub, Postgres, Jira and MCP).

## Status

Beta. The engine has had three adversarial review passes, a full CodeRabbit sweep, a public-benchmark
campaign and one live production workload. The verdict logic is stable; what is still moving is
deployment hardening (key distribution, a shared replay store, TLS, multi-instance) and the ergonomics
people ask for. **If you run an agent that writes to a real system, the most useful thing you can do is run
Corrobo in monitor mode beside it for a week and tell the author what it found.**

## Feedback

Bugs, false verdicts, and "it should have caught this" reports are the most valuable thing you can send.
Write to the author through the contact on the PyPI page, and include the `explain()` output and the
evidence entry (both are secrets-free by construction).

## License

PolyForm Noncommercial 1.0.0. Free to use, copy, modify and share for noncommercial purposes: personal
projects, research, teaching, and noncommercial organizations. **Commercial use needs a separate license:**
contact the author. See `LICENSE` for the full terms.
