Metadata-Version: 2.5
Name: qa-orchestrator
Version: 0.3.0
Summary: Inject faults into a substrate, drive captures, and check the referee reached the verdict it should have.
Project-URL: Homepage, https://github.com/james-sheen/qa-orchestrator
Project-URL: Source, https://github.com/james-sheen/qa-orchestrator
Project-URL: Issues, https://github.com/james-sheen/qa-orchestrator/issues
Author: James Sheen
License-Expression: Apache-2.0
License-File: LICENSE
License-File: NOTICE
Keywords: certification,fault-injection,plugin,qa,test-harness
Requires-Python: >=3.10
Requires-Dist: pyyaml>=6
Provides-Extra: bmc
Requires-Dist: bmc-sensor-audit[detect]<0.3,>=0.2.0; extra == 'bmc'
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == 'dev'
Description-Content-Type: text/markdown

# qa-orchestrator

Inject faults into a substrate, drive captures, and check that the referee — the program
being graded — reached the verdict it should have. The harness knows nothing about any
domain: tiers, verbs, referees and report shapes are registered by **verticals**, and the
first vertical (`bmc-sensor-audit` over BMC firmware) is one of them, loaded through the
same door as any other.

```
pip install -e '.[dev]'

qa-orchestrator list           # what this process can name

# The first vertical's scenarios live with the first vertical.
qa-orchestrator check src/qa_orchestrator/verticals/bmc_scenarios/
qa-orchestrator run   src/qa_orchestrator/verticals/bmc_scenarios/sensor-removed.yaml

PATH=examples/paper/bin:$PATH \
qa-orchestrator --plugin examples/paper/vertical.py run examples/paper/withdrawn.yaml
```

The last line grades a proposal's requirements with a checker that has never seen a BMC:
positional arguments where the first vertical's tool takes flags, `issues` where it
keeps `findings`, and a report of what it did **not** check, which the scenario asserts on.

**Released — 0.3.0**, tagged `v0.3.0`, Apache-2.0, on PyPI as `qa-orchestrator`.

**0.3.0 is the domain-free rewrite.** Tiers, verbs, referees and report shapes come
from registries and the core names none of them; the BMC tool, its tiers and its
scenarios are a vertical loaded through an entry point, like any other would be. A
`qa-scenario/1` file written for 0.2.x still reads and still runs.

Exit codes are the referee's own three: `0` every expectation held, `1` a verdict
disagreed, `2` the run could not be completed. `2` never reads as clean.

[`DESIGN.md`](DESIGN.md) says what was locking the harness to one domain, where each lock
went, the contract a vertical must meet, the scenario format, and how to migrate the
0.2.x repository.

Apache-2.0.
