---
id: tests
paths:
  - "tests/**"
description: Test policy and coverage requirements
---

<!-- rule-version: 0.26.1 -->

# Test Policy (canonical)

> **Rule version:** `0.26.1` — the full-suite test invocation is `uv run gz test`, the parallel canonical runner; the serial `-m unittest` form is retired (operator ruling 2026-09-18, under GHI #921). Prior `0.26.0`: diet pass under GHI #921: measured instances, worked examples and history lifted to [Tests — Rationale](../../docs/governance/tests-rationale.md); prior versions in [Rule Version History](../../docs/governance/rule-version-history.md#testsmd). Binding rules unchanged.

## General Rules (binding)

- Use **stdlib `unittest`**; no pytest — no pytest syntax, fixtures, parametrization or plugins. `unittest.mock` and `tempfile.TemporaryDirectory()` are the fixtures.
- Table-driven tests with deterministic seeds; no network or external services; fixtures local, small and reproducible.
- **Smoke/BVT <=60s** — binds the `@smoke`-marked subset run by `uv run gz smoke`, NOT the full unit tier. That verb fails closed (exit 3) on breach or an empty tier and runs as the `Smoke tier` step of `gz check`. The full unit tier has no ceiling: it grows with the REQ set, and the canonical "Tests pass" invocation runs it with `unittest-parallel`.
- Unit tests use `tempfile` temp DBs; never a live database.
- **Git fixture isolation:** every `git` a test spawns passes `env=_isolated_git_env()` from `tests/commands/common.py`, and `tests/__init__.py` drops the same repo-local variables once, so a fixture targeting a temp repository operates on that repository whatever the process inherited — from a linked worktree an unguarded `git init` re-initialises the hosting repo as bare (GHI #977). Fenced by `tests/commands/test_common_fixtures.py::TestEveryGitSpawnIsInsideTheBoundary`.
- Cleanup through `tempfile.TemporaryDirectory()`; never raw `shutil.rmtree()` in tearDown.

## Coverage Floor (binding)

- Coverage >=40.00% (the `coverage-40pct` chore holds the enforced value); verify it has not regressed before closing a brief.

## Run / Verify

```bash
uv run ruff check . --fix && uv run ruff format .
uv run gz test
```

All tests pass; `uv run gz smoke` exits 0; coverage at or above the floor.

### Smoke tier membership (binding)

A smoke test proves a surface answers at all — build verification, not correctness. Mark it `@smoke` from `gzkit.smoke`; there is no smoke directory (`gz validate --test-tiers` forbids a third tier). Prefer members that enumerate coverage from a live source (the CLI parser, the registry) over a hand-maintained list, and keep them fast and free of subprocess work.

## Red-Green-Refactor (TDD Discipline — binding)

Gate 2 is named TDD. Per behavior increment:

- **Red:** write a test for a behavior the OBPI brief requires (`REQ-*`), run it, and watch it fail for the right reason. A test that passes on first run is not Red.
- **Green:** the simplest code that makes it pass.
- **Refactor:** improve structure without changing behavior; new behavior starts a new Red.

**Derivation rule:** test cases derive from the brief's acceptance criteria, not from the implementation; outside a pipeline run, locate the governing brief first.

**Tests assert semantics, not strings (invariant 6f).** Assertions derive from the REQ or skill contract, not from a run of the code. A unit test is a `unittest.TestCase` check of one required behavior — fast, isolated, deterministic — that fails when the contract breaks; it does not preserve implementation structure, prose or rendered output unless that output form is the named contract.

**The discriminator:** *if the production code's behavior changed but its text did not, would this test fail?* If not, it is not a BEHAVIOR proof — a test that greps a doc or asserts a file exists proves content; route it to the SUPPORT channel or delete it. **Prefer structured assertion targets** — fields, domain objects, state transitions, exception types, ledger event fields, parsed values — and assert exact strings only where rendering is the named contract.

**Eval-awareness corollary (advisory).** Audit-helper names SHOULD NOT pattern-match as audit-step names; name them by behavior (`assert_receipt_id_resolves`, not `assert_audit_passes`).

**Output-form fixture carve-out.** Output-form assertions are permitted in dedicated fixture tests per `.gzkit/rules/tool-skill-runbook-alignment.md` § Invariant 3, in classes separate from REQ-derived tests. Declare the carve-out with an `# output-contract: <reason>` comment in the test or a class name ending `OutputForm`, `OutputContract` or `Rendering`; `gz test-shape` reads the markers and reports an undeclared `result.output` / `.getvalue()` / `assertRegex` assertion as advisory (GHI #571) — advisory is the settled disposition. Output-form fixture tests are BEHAVIOR proofs under § REQ Scope Discipline: they test render code, not file content.

**Per-increment rhythm:** one test → one observed RED → minimum code to GREEN → next increment. **Do not slice horizontally.** Authoring every test for a brief and then every implementation is not TDD with a long cycle — it produces tests insensitive to change, none of which was observed failing for the right reason.

**RED evidence:** Do not author ARB *step* receipts with `exit_status=1` as "RED receipts". Gate 2 claims cite GREEN-side step receipts; the RED witness is `uv run gz arb red --req <REQ-ID>`, which runs the covering test with the production hunks withheld and emits an `arb-red-<REQ>` receipt whose `failure_class` `none` means the test cannot fail (GHI #642).

**Mutation-sweep integrity (binding, GHI #963).** A failing mutant run is NOT a kill. Report four outcomes: `killed` and `survived` are claims about the guard, `invalid` and `inconclusive` about the run. Every mutant runs in a subprocess with its own `PYTHONPYCACHEPREFIX` (CPython validates a cached `.pyc` on mtime-seconds and size, so two equal-length mutants in one second share bytecode). Use `gzkit.mutation_witness.run_mutation_sweep`, name the covering test via `Mutation.expected_tests`, and cite a sweep only when `is_conclusive` is true, disclosing `invalid` and `inconclusive` rows.

**Verification exit-code integrity (binding, GHI #589).** A verifier's truth is its own exit code, never a downstream filter's. Capture to a file (`verifier > out.log 2>&1`) and read `$?` in the statement immediately after; for attestation cite the ARB receipt's `exit_status`. The shell reports the last thing it ran, so each of these masks a failure: a pipe (`verifier | tail`; only `set -o pipefail` or `${PIPESTATUS[0]}` preserves it), a following statement (`verifier > log; tail log`; only `set -e` or an immediate `$?` read), an `||` branch (`verifier || echo failed` exits 0 exactly when the verifier failed — write `verifier || echo "REAL EXIT: $?"`), a group (`( … )` and `{ …; }` report their last statement), a chain whose final separator is not the verifier's, and `&` (reports the launch). The aggregate status is a floor, not a proof (GHI #969). Mechanized by the `verifier-pipe-gate.py` PreToolUse hook over `Bash`, whose verifier set is read from `CANONICAL_STEP_COMMANDS`.

## TASK-Driven Workflow (binding)

TASK is the leaf of `PRD → Constitution → ADR → OBPI → REQ → TASK → Attestation`. **src/tests commits MUST carry a `Task:` trailer.** Enforced by `gz validate --commit-trailers`. `Ceremony:` and `Eval-feedback-source:` do not substitute on src/tests scope.

| Form | Use | Example |
|------|-----|---------|
| `TASK-X.Y.Z-NN-MM-PP` | OBPI-scoped work, minted by `gz task start` | `Task: TASK-0.22.0-01-01-01` |
| `TASK-<kebab-slug>` (optional `-#<ghi>`) | Direct-fix work; append the GHI only when one already exists — never file one to satisfy the trailer (operator directive 2026-06-01) | `Task: TASK-fix-git-log-encoding` |
| `TASK-gz-git-sync` | Auto-stamped by `gz git-sync` | `Task: TASK-gz-git-sync` |

OBPI-scoped: `gz covers` → `gz task start` → TDD cycle → commit with `Task:` → `gz task complete` → `@covers(REQ-...)`. Direct-fix: precedent check (`git log --since='60 days ago' --oneline --grep='^fix('`) → TDD cycle → commit with `Task: TASK-<slug>`.

## Two runners, one test surface

| Runner | Location | Purpose | Contract |
|--------|----------|---------|----------|
| `unittest` | `tests/` | Pure Python behavior, command contracts | Mocked subprocess boundaries (`_uv_sync_patcher`, `_git_subprocess_patcher`, `_quick_init` in `tests/commands/common.py`); deterministic; < 200ms; `tempfile` DBs |
| `behave` | `features/` | End-to-end CLI and governance scenarios | Real operator flows; anything needing a real subprocess lives here |

`gz test` runs `unittest` then `behave`; both must pass for `gz check`.

**Behave scenarios covering a REQ carry `@REQ-X.Y.Z-NN-MM`** as a scenario tag — `gz validate --behave-req-tags` (fires on `Completed`/`Validated` briefs; waivers in `data/behave_coverage_waivers.json`). The gate is REQ-kind-aware (GHI #636): a BEHAVIOR REQ is satisfied by a tagged scenario or an `@covers` unit test; SUPPORT and STRUCTURAL-FENCE REQs are exempt by kind; a missing `[kind]` tag defaults to BEHAVIOR.

## REQ Scope Discipline (binding)

Every REQ in an OBPI brief's `## Acceptance Criteria` carries exactly one kind as an inline tag — `REQ-X.Y.Z-NN-NN [kind]: claim` — author UPPERCASE, do not rewrite existing lowercase tags (operator ruling 2026-08-11). `gz validate --req-kind-discipline` enforces it (ADR-0.0.59).

| Kind | Covers | Proof channel |
|------|--------|---------------|
| **BEHAVIOR** | Code behavior — functions, commands, CLI output, state transitions | `@covers` test in `tests/**` |
| **SUPPORT** | Artifacts that support behavior — docs, rules, data files | The cited artifact present (a ledger event citing the path, or on disk for `artifact_edited`) AND its structural validator scope admits it |
| **STRUCTURAL-FENCE** | Cross-OBPI invariants scoped to the parent ADR | Parent-ADR `## Boundary Invariants` entry naming this OBPI (`(OBPI-NN[, …])`), audited at ADR closeout |

**Do not add a unit test to make a SUPPORT or STRUCTURAL-FENCE REQ appear covered** — that is the tautological content test this taxonomy exists to remove (GHI #571).

> Canonical expansion, quantification and the GHI #270 reconciliation: [`docs/governance/req-scope-discipline.md`](../../docs/governance/req-scope-discipline.md). TDD anti-patterns, the two-runner history, the exit-code measured instances, the mutation-sweep example and the TASK workflow details: [`docs/governance/tests-rationale.md`](../../docs/governance/tests-rationale.md).
