This package uses a Rack suite under tests/ as the primary
test architecture. The strategy covers the package boundary for source,
documentation, contracts, generated governance output, and release-facing
checks. Workspaces that aggregate multiple packages should keep one
equivalent strategy document at each audited package boundary and may add a
workspace overview when cross-package signoff matters.
Rack is the suite manifest and execution model. The suite manifest at
tests/rack.toml declares ordered strata,
and each stratum declares subtests in STRATUM.toml. The
L99_signoff stratum is the release-facing gate and includes
the project audit, contract checks, documentation checks, repository
hygiene checks, lockfile validation, and packaging-signoff expectations.
The reference package is a pure Python tool, so it does not currently have separate Python/C++ parity lanes. Projects with multiple runtimes, such as Python wrappers over native code or browser/WASM lanes, should document which public surfaces are tested in each lane, how parity is established, and which gaps are tracked as accepted exceptions or follow-up work.
This repository has a small fixture footprint and no custom oracle generator. Larger adopters should describe fixture data, generated assets, captured cases, golden files, oracle tools, and any asset-discovery checks that prevent missing or orphaned test material from silently falling out of coverage.
Coverage evidence is intentionally layered. Rack manifests explain the executable suite shape, dev-std governance records link requirements and ADRs to verification refs, and audit scopes catch missing manifests, missing documentation, stale links, and orphan plan/log material. Known gaps should be visible as issue refs, exceptions, or explicit release signoff notes rather than implicit omissions.
Plans that add or modify tests must use the
test-runtime-impact-audit closeout step to list those tests,
identify their Rack strata or files, and record observed runtime impact
where available. New minute-scale tests should receive a reasonable
optimization pass or be split into an appropriate slower signoff lane so
routine development loops do not grow by accident.
The fast foundation lane contains TypeSpec policy tests
(L0_025), application-boundary tests (L0_026),
and template command tests (L0_027). These tests use temporary
repository shapes and do not install Node dependencies. Public capability
summaries and CLI inventory extend L0_001 and
L0_002; config-schema parity and shipped-template config
validation extend L99_002.
Clean-copy executable validation is intentionally isolated in
L99_006. It installs the lockfiles for both shipped templates,
audits each copied project, and runs its complete signoff. The observed
local runtime was approximately 60 seconds total (about 30 seconds for
each template). Keeping this work in L99_signoff preserves the
subsecond focused policy loop
while making template drift release-blocking.
The Lit package lane contains nine Vitest files and 38 tests covering the
activity engine and lifecycle edges, browser history, DOM hosting, shell
profiles, semantic clients, protocol selection, and background cleanup.
The TypeSpec package lane compiles the contract, checks generated
freshness, compiles a generated TypeScript consumer, and runs shared
Node/Ajv and Python/jsonschema conformance vectors. The final full Rack run
observed 113 seconds for L0_foundation and 96 seconds for
L99_signoff; the complete run was approximately 217 seconds.
uv run rack run L0_foundation
uv run rack run L99_signoff
uv run dev-std audit . --scope docs.test_strategy
uv run dev-std audit . --scope tests