Metadata-Version: 2.5
Name: afterstate
Version: 0.1.0
Summary: Deterministic external-state invariant testing for AI agents: verify what actually changed after your agent acts.
Project-URL: Repository, https://github.com/Createyouracccount/afterstate
Author: Createyouracccount
License: MIT
License-File: LICENSE
Keywords: ai-agents,ci,fault-injection,idempotency,invariants,testing
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.11
Requires-Dist: httpx>=0.27
Requires-Dist: pyyaml>=6.0
Provides-Extra: dev
Requires-Dist: fastapi>=0.110; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: uvicorn>=0.29; extra == 'dev'
Provides-Extra: postgres
Requires-Dist: psycopg[binary]>=3.1; extra == 'postgres'
Description-Content-Type: text/markdown

# afterstate

[![ci](https://github.com/Createyouracccount/afterstate/actions/workflows/ci.yml/badge.svg)](https://github.com/Createyouracccount/afterstate/actions/workflows/ci.yml)
[![sandbox-loop](https://github.com/Createyouracccount/afterstate/actions/workflows/sandbox-loop.yml/badge.svg)](https://github.com/Createyouracccount/afterstate/actions/workflows/sandbox-loop.yml)

**Deterministic external-state invariant testing for AI agents.**

Your agent said "done." What did it actually *do* to your database, your
payments, your inventory? `afterstate` doesn't grade the agent's answers or its
tool-call trajectory — it attacks the systems your agent touches with
duplicate requests, client retries, conflicting idempotency keys and duplicate
webhooks, then verifies hard state invariants: money moved exactly once, a
decline left no trace, a compensation actually restored the balance.

No LLM judge anywhere. Same inputs, same verdict, every run.

```
[PASS] retry-after-timeout-grants-once
[FAIL] duplicate-request-grants-once
       one_effect_per_key: duplicate effects in credit_ledger: dup-k1×2
       balance_delta: delta is 500 (before=0, after=500), expected 250
```

## Why

Eval tools compare messages and tool-call trajectories. But the failures that
cost real money happen one layer down, in the state your agent leaves behind:

- a timeout retry charges the customer twice
- a declined payment still writes a ledger row
- a refund exceeds the original charge
- a webhook is emitted twice and the consumer double-fulfills

Every one of these can look like a **200 OK with a perfectly reasonable
transcript**. You only catch them by diffing external state under fault
injection — which is what `afterstate` does, in CI, deterministically.

## Install

Requires Python 3.11+.

```bash
pip install afterstate                # once v0.1.0 lands on PyPI
pip install git+https://github.com/Createyouracccount/afterstate  # until then
```

The demo service and its contracts live in the repo, not the wheel — clone
for the full experience.

## Ninety-second demo, one command

The repo ships a deliberately realistic demo: a SaaS credit-issuance service
whose buggy mode contains four production-grade bug classes (ignored
idempotency keys, declines with side effects, overdrawable balances,
double-emitted webhooks).

```bash
git clone https://github.com/Createyouracccount/afterstate && cd afterstate
./demo.sh                     # or: docker compose run --rm demo
```

`demo.sh` deploys the buggy service, attacks it with 7 money-safety contracts
(6 fail, each with the exact duplicated ledger rows or wrong deltas), then
deploys the fixed service and shows the same contracts go green. A daily
[sandbox-loop](.github/workflows/sandbox-loop.yml) workflow re-runs this
red→green proof in CI and publishes the dated reports — the demo cannot
silently rot. See a [sample failure report](docs/sample-report.md).

## Your own system

```bash
afterstate init               # scaffolds contracts/starter.yaml
afterstate run contracts/starter.yaml
```

Point `target` at a **sandbox** deployment of your service and its database,
list the tables your money flows through, pick faults and invariants. Guides:

- [Writing your first contract](docs/first-contract.md)
- [Full contract reference](docs/contract-reference.md)

## How it works

A **contract** is a YAML file describing one scenario:

```yaml
name: retry-after-timeout-grants-once
target:
  base_url: http://127.0.0.1:8123
  db: sqlite:///credits.db          # or postgresql://...
setup:
  - http: { method: POST, path: /admin/reset }
capture:
  tables: [credit_ledger, balances]
invoke:
  - http:
      method: POST
      path: /credits/grant
      json: { user_id: u1, amount: 100 }
      headers: { Idempotency-Key: retry-k1 }
    faults: [retry_after_failure]    # timeout, then client retry
    expect_status: [200]
assert:
  - one_effect_per_key: { table: credit_ledger, key: idempotency_key }
  - balance_delta:
      query: "SELECT COALESCE(SUM(amount),0) FROM balances WHERE user_id='u1'"
      delta: 100
```

The runner executes `setup → snapshot → invoke (with faults) → snapshot →
assert → cleanup` and reports per-invariant verdicts.

### Fault injection

Request faults go under `invoke[].faults`:

| fault | what it simulates |
|---|---|
| `duplicate_request` | the same request delivered twice (double click, queue redelivery) |
| `response_lost` | server processed, client never saw the response (timeout / reset) |
| `retry_after_failure` | the classic timeout-then-retry under one idempotency key |
| `conflicting_payload_same_key` | an idempotency key reused with a different payload |

Webhook delivery faults go under the contract's `webhooks.faults` block (which
also runs the inbox that records what your service emits — see the
[contract reference](docs/contract-reference.md)):

| fault | what it simulates |
|---|---|
| `duplicate_webhook` | each webhook delivered twice to the consumer |
| `out_of_order_webhooks` | webhook delivery order reversed |

### Built-in invariants

| invariant | asserts |
|---|---|
| `one_effect_per_key` | at most one effect row per idempotency key |
| `no_effect_on_failure` | failed/declined calls leave zero state changes |
| `sums_equal` | two SQL aggregates agree (ledger vs balances, order vs payment) |
| `balance_delta` | a quantity changed by exactly the expected amount |
| `nonnegative` | a column never goes below zero (inventory, balances) |
| `state_restored` | a compensating flow returned tables to their prior state |
| `webhook_exactly_once` | each event key delivered exactly once |

## CI

`afterstate run` exits nonzero on any failure and emits JUnit XML:

```yaml
# .github/workflows/afterstate.yml
name: afterstate
on: [push]
jobs:
  invariants:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "3.12" }
      - run: pip install afterstate
      - run: ./scripts/start-sandbox.sh   # your sandbox SUT
      - run: afterstate run 'contracts/*.yaml' --junit afterstate.xml --report report.md
      - uses: actions/upload-artifact@v4
        if: always()
        with: { name: afterstate-report, path: report.md }
```

## Scope (v0.1)

- HTTP systems under test; SQLite and PostgreSQL state capture; webhook inbox.
- `afterstate` acts as the client and injects faults client-side — which is
  deterministic and covers the client-visible half of timeouts, resets and
  redeliveries. A transparent proxy that intercepts a live agent's own traffic
  is the v0.2 extension point.
- Not an agent framework, not an LLM evaluator, no dashboard. It is a testing
  engine you point at a sandbox.

## License

MIT
