Metadata-Version: 2.4
Name: algovoi-atb-vnext
Version: 0.1.0
Summary: ATB vNext: a verifiable, offline, multi-mode evaluation harness for payment agents
License: Apache-2.0
Project-URL: Homepage, https://agent-trust-bench.algovoi.co.uk
Project-URL: Documentation, https://docs.algovoi.co.uk/atb-vnext
Keywords: agent evaluation,payment agents,AI safety,OWASP LLM,keystone,RFC 8785
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Requires-Dist: rfc8785>=0.1.2
Requires-Dist: cryptography>=42.0
Dynamic: license-file

# ATB vNext (engine)

A verifiable, offline, multi-mode evaluation harness for payment agents. Built on the live ATB
Keystone binding: every step, score, and verdict is a content-addressed reference
(`sha256:` + SHA-256(RFC 8785 JCS)), so a result recomputes byte-for-byte and is Falcon-signable,
exactly like the ATB Pass Certificate.

## What is here (Phase 1, staged, local)

- `atb_vnext/refs.py` — the trace model: `step_ref` (alias execution_ref) -> `journey_ref`
  (alias trust_query_ref) -> `run_ref` (`atb_run_v2`, the signed/diffable unit) + `score_ref`.
- `atb_vnext/schema.py` — `atb-fixture-1` strict fixture schema + dependency-free validator.
- `atb_vnext/graders.py` — the five scores: outcome, trace, trust (judge hook), cost_latency,
  and the headline `severity_weighted` (unsafe success penalised by blast radius).
- `atb_vnext/runner.py` — `run_fixture(fixture, transcript)` -> recomputable `atb_run_v2`.
- `atb_vnext/packs/` — priority-one fixtures: `tool-misuse` (3) + `aml-edgecase` (1).
- `tests/` — 16 conformance tests (schema, determinism, byte-for-byte recompute, tamper
  divergence, severity weighting, REFER-as-safe-block, judge hook). All pass.
- `goldens/` — per-fixture golden `run_ref` baselines (nightly regression anchor).

## Modes and scores

See `DESIGN.md` for the full matrix (unit / workflow / persona / redteam / runtime / regression),
the six scenario packs, the fixture schema, and the pass/fail rules.

## Run

```bash
PYTHONPATH=. python -m pytest tests/ -q
```

## Full feature set (all built + tested; live on VM4)

- **6 modes:** unit, workflow, persona, redteam, runtime, regression.
- **7 packs, 15 fixtures:** tool-misuse, aml-edgecase, persona, long-context, multi-agent,
  data-protection, runtime. All 7 OWASP LLM Top-10 categories the bench maps are covered.
- **5 scores:** outcome, trace, **trust** (deterministic `RubricJudge`, or plug an `LLMJudge`),
  cost_latency, and the headline severity_weighted.
- **Agent-driven run loop** (`client.py`): `run_agent` / `run_and_report` drive any `AgentAdapter`
  through the fixtures, threading conversation history so stateful agents are testable. Reference
  adapters in `agents.py` (`HeuristicPolicyAgent`, `NaiveProceedAgent`, `LiteralInjectionAgent`).
- **Runtime kill-switch** (`runtime.py`): scores in-flight, halts on a severity breach.
- **Replay from production** (`replay.py`): turn a real `bench_events` session into a fixture.
- **Dynamic persona simulation** (`persona_sim.py`): an adversary that escalates on the agent's reply.
- **Mutation testing** (`mutate.py`): obfuscate an attack (base64 / zero-width / leetspeak) to
  expose brittle detection.
- **Suite analysis** (`analysis.py`): coverage report, flakiness detector, regression diff,
  cross-cutting safety invariants, differential agent comparison.
- **Configurable risk policy** (`RiskPolicy`): per-deployment severity weights + tolerances.
- **Signed corpus** (`corpus/`, verify with `verify_corpus.py`): 15 positives + 15 negatives,
  Ed25519-signed under the same trust root as the main JCS corpus.
- **Live API** (`/agent-trust-bench/vnext/`): `packs`, `coverage`, `fixture/{id}`,
  `score` (returns a Falcon-1024-signed, recomputable `atb_run_v2` with the trust score populated).
