Metadata-Version: 2.4
Name: banza-conformance
Version: 0.1.0
Summary: BANZA protocol conformance runner — generates conformance evidence (not certification) for an operator endpoint.
Author: BANZA Protocol
License: Apache-2.0
Project-URL: Homepage, https://banza.org/conformance
Project-URL: Source, https://github.com/banza-protocol/banza
Keywords: banza,conformance,protocol,evidence
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Provides-Extra: federation
Requires-Dist: cryptography>=41.0.0; extra == "federation"

# Banza Conformance Runner

Tests a running Banza operator against the protocol conformance suite and produces a structured JSON report.

> The runner generates **conformance evidence, not certification**. A PASS is technical
> conformance evidence for the requested level — it is **not** a production certificate.
> Production certification remains M2/M3-gated and is issued separately by the BANZA CA.

## Distribution for external operators

External operators can run BANZA conformance against their **public sandbox endpoint**
without cloning this repository, in three ways.

### Installable CLI

```bash
pip install banza-conformance        # from tools/banza-conformance
banza-conformance --url https://sandbox.operator.example --level 0 --output report.json
```

Supported options: `--url` (required), `--level`, `--suite`, `--output`, `--quiet`,
`--federation` (L3, requires the `federation` extra: `pip install "banza-conformance[federation]"`).
The CLI calls the same conformance logic as `run.py` — `python3 tools/banza-conformance/run.py …`
continues to work unchanged.

On completion the tool prints:

```
Conformance report generated. This is evidence, not certification.
```

### Client identity (User-Agent)

The runner sends a stable, documented User-Agent so operators can allow it in
WAF / proxy / CDN configurations:

```
BANZA-Conformance/1.0 (+https://banza.org/conformance)
```

If your sandbox endpoint sits behind a bot filter (e.g. Cloudflare), allow this
User-Agent (or the `/health` and `/.well-known/banza/*` paths) so the runner can reach it.
The User-Agent identifies the client cleanly; it does not bypass any security control.

### Docker

```bash
docker run --rm \
  -v "$PWD/reports:/reports" \
  ghcr.io/banza-protocol/banza-conformance:latest \
  --url https://sandbox.operator.example \
  --level 0 \
  --output /reports/banza-l0-report.json
```

Build the image locally (context = this directory):

```bash
docker build -t banza-conformance:local tools/banza-conformance
```

The entrypoint is `banza-conformance`; arguments after the image name are passed
straight through. Reports are written under `/reports` — mount a host volume there.

### GitHub Action

Run BANZA conformance in your CI/CD against your public sandbox endpoint:

```yaml
name: BANZA Conformance

on:
  workflow_dispatch:
  pull_request:
  push:

jobs:
  conformance:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: banza-protocol/banza/.github/actions/banza-conformance@main
        with:
          url: https://sandbox.operator.example
          level: "0"
          output: banza-l0-report.json
      - uses: actions/upload-artifact@v4
        with:
          name: banza-conformance-report
          path: banza-l0-report.json
```

Inputs: `url` (required), `level` (default `"0"`), `output`
(default `"banza-conformance-report.json"`), `suite`, `federation`, `quiet`.

## Requirements

Python 3.8+. No third-party packages for the core runner — stdlib only.

For L3 federation conformance (signature verification): `pip install cryptography>=41.0.0`

## Usage

### Standard conformance (L0–L4)

```bash
# Run all suites against a local operator
python3 tools/banza-conformance/run.py --url http://localhost:3000

# Run up to Level 2 only
python3 tools/banza-conformance/run.py --url http://localhost:3000 --level 2

# Run a single suite
python3 tools/banza-conformance/run.py --url http://localhost:3000 --suite health

# Write report to file (quiet mode)
python3 tools/banza-conformance/run.py --url http://localhost:3000 --output report.json --quiet
```

Or via the shell wrapper:

```bash
./tools/banza-conformance/run.sh --url http://localhost:3000 --output report.json
```

### L3 Federation Conformance (79 tests, 9 suites)

The federation conformance suite tests cross-operator federation protocol compliance.
It requires two terminals: one for the fixture server (Operator A adapter) and one for the runner.

```bash
# Terminal 1 — start the fixture server (Operator A)
python3 tools/banza-conformance/fixture_server.py --port 8099

# Terminal 2 — run the full federation suite (79 tests)
python3 tools/banza-conformance/run.py \
  --federation \
  --url http://localhost:8099 \
  --output l3-report.json

# Run only one federation sub-suite
python3 tools/banza-conformance/run.py \
  --federation \
  --url http://localhost:8099 \
  --fed-suite cert        # cert | disc | trust | route | exec | obl | evt | settle | fail
```

**What the federation runner does:**
- Generates an ephemeral test BANZA root keypair (never reused as production)
- Spins up an embedded Simulated Operator B (random port, in-process)
- Spins up an embedded test BRL/key-manifest server (random port, in-process)
- Runs all 79 federation tests against the fixture server
- Produces a structured JSON evidence report

**Federation suites:**

| Suite | Cases | Blocking | What it tests |
|---|---|---|---|
| FED-CERT | 11 | Yes | Certificate schema, signature, expiry, BRL |
| FED-DISC | 8 | Yes | Federation manifest extension fields |
| FED-TRUST | 9 | Yes | All 9 ADR-026 trust protocol steps |
| FED-ROUTE | 12 | Yes | Routing wire protocol, idempotency, rejections |
| FED-EXEC | 8 | Yes | Acceptance semantics, ledger atomicity |
| FED-OBL | 7 | Yes | Obligation creation, signature, state machine |
| FED-EVT | 6 | No | Federation event emission and schema |
| FED-SETTLE | 10 | No | Netting, settlement, reconciliation |
| FED-FAIL | 8 | No | Retry, crash recovery, revocation, mismatches |

**L3 certification decision:** All blocking suites (FED-CERT through FED-OBL) must PASS. FAIL in a blocking suite denies certification. FAIL in a non-blocking suite produces conditional certification (30-day remediation window).

See `docs/federation/FEDERATION_CONFORMANCE_MODEL.md` for the full decision rules.

## Options

| Flag | Description |
|------|-------------|
| `--url` | Base URL of the operator (required) |
| `--level` | Maximum certification level to test, 0–4 (default 4) |
| `--suite` | Run one suite only: `health`, `wallets`, `transfers`, `traces`, `manifest` |
| `--output` | Write JSON report to this path (default: stdout) |
| `--quiet` | Suppress PASS lines; only print failures and summary |

## Certification levels

| Level | Name | Runner suites | Awarded here |
|-------|------|---------------|--------------|
| L0 | Protocol Sandbox | health, manifest (sandbox-safety) | Yes |
| L1 | Core Payment Capability | + wallets, transfers, traces (traceability) | Yes |
| L2 | Payment Initiation Capability | + payment_initiation (payment requests, dynamic QR, instant execution) | Yes |
| L3 | Inter-Operator Interoperability | federation suite — run via `--federation` (`run_fed.py`) | No (multi-operator) |
| L4 | External Interoperability | External Interoperability profile (external-rail evidence) | No (profile) |

Level names **and per-level capabilities** are canonical and match
`BANZA_CERTIFICATION.md` § Conformance level model and ADR-034. This
single-operator sandbox runner awards **L0–L2**; L3 (federation) and L4 (external)
require multi-operator / external-rail evidence and are not awarded here.

## Exit codes

| Code | Meaning |
|------|---------|
| 0 | All executed cases passed |
| 1 | One or more cases failed |
| 2 | Runner error (connectivity, bad arguments) |

## Report format

Reports conform to `conformance/report-schema.json`. Each case records:
- `status`: PASS / FAIL / SKIP / ERROR
- `request` and `response` for HTTP cases
- `assertions`: per-field check results
- `failure_reason`: human-readable explanation on failure

## Adding new suites

1. Implement a runner function `run_<name>(base_url, ctx) -> list[CaseResult]`
2. Register it in the `SUITES` dict
3. Add it to the relevant `LEVEL_SUITES` entries
4. Add the corresponding vectors in `conformance/vectors/`
