Metadata-Version: 2.4
Name: spec-trace
Version: 0.1.2
Summary: Traceability lint between behavioural specs and tests
Project-URL: Repository, https://github.com/sminnee/spec-trace
Project-URL: Issues, https://github.com/sminnee/spec-trace/issues
Author: Sam Minnee
License: MIT
License-File: LICENSE
Keywords: lint,spec-driven-development,specs,testing,traceability
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.11
Provides-Extra: dev
Requires-Dist: pyright>=1.1; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Requires-Dist: vulture>=2.11; extra == 'dev'
Description-Content-Type: text/markdown

# spec-trace

A traceability lint between behavioural specs and tests. Zero dependencies, one command, works as
a CI gate.

Spec points are markdown bullets that end in a keycode. Tests cite the keycodes they cover.
`spec-trace` reads both sides and reports the drift.

```markdown
- Results can be filtered to a date range. Both boundaries are inclusive. `[date-filter-range]`
```

```python
# SPEC: warehouse/querying#date-filter-range
def test_filters_by_date_range() -> None: ...
```

```console
$ spec-trace
OK
```

## Install

```bash
uv tool install spec-trace     # or: uvx spec-trace
```

Python 3.11+. No runtime dependencies — the stdlib-only single package is the distribution story.

## Getting started

```bash
spec-trace init     # scaffold specs plus the agent instructions
spec-trace          # lint
spec-trace docs     # the full convention
```

`init` writes a specs directory with a landing page, a Claude Code skill, and an `AGENTS.md`
section. The convention is more than half the product: the checker is useless without the writing
discipline, and its main consumer is coding agents.

Two flags change what it writes. `--example` adds a starter spec file to copy the format from; it
is opt-in because no test cites its keycodes, so `spec-trace --warn` reports them as coverage gaps
until you replace it. `--skill-only` writes the skill alone, for a repo that already has its own
specs. The two are mutually exclusive, as is `--global`.

## What the gate fails on

`spec-trace` hard-fails on referential integrity only:

| Failure | Meaning |
| --- | --- |
| Orphaned ref | a test cites a spec ID that does not exist |
| Malformed ref | a `SPEC:` token with no `#keycode` |
| Duplicate ID | the same keycode twice in one spec file |
| Over-long bullet | behaviour text above the limit (default 100 chars) |

Coverage gaps — untested spec points and unspecified tests — are advisory, shown under `--warn`.

**This split is the reason the tool survives as a gate.** The failures above are cheap to fix and
objectively wrong. A gate that failed on coverage would be deleted within a month. In the codebase
this tool was extracted from, the gate has stayed green since it was enforced in CI, across 167
spec files and 3,602 spec points.

Orphaned refs get a fuzzy-match suggestion, because a mistyped keycode is the most common failure:

```
ERROR: 1 orphaned test references
  A test cites a spec ID that does not exist. Fix the ID, or add the spec point.
  tests/querying_test.py:
    - SPEC: warehouse/querying#date-filter-rnage (line 42)
        did you mean `warehouse/querying#date-filter-range`?
```

## Usage

```bash
spec-trace                    # lint the repo; exit 1 on a gate failure
spec-trace warehouse/         # limit the report to one subsystem
spec-trace --warn             # include advisory coverage sections
spec-trace --warn-unspecified # as --warn, plus a list of each unspecified test
spec-trace --missing          # list untested spec points
spec-trace --ok               # list tested spec points
spec-trace --orphan           # list orphaned references
spec-trace --json             # machine-readable output
spec-trace --root path/to/repo   # scan this repo root instead of auto-detecting
spec-trace --version          # print the version
spec-trace init --update      # refresh generated instruction files
spec-trace init --example     # also write a starter spec file
spec-trace init --skill-only  # write only the Claude Code skill
spec-trace init --global      # install the skill into ~/.claude instead
```

`--warn-unspecified` implies `--warn`: it prints the advisory sections *and* names each
unspecified test, where `--warn` alone reports only how many files contain them. Reach for it when
the summary count is not enough and you need to act on the individual tests.

`--root` skips the auto-detection that walks upward for a config file or a `.git` directory. It is
what you want when the tool runs from outside the repo it should scan. `init` accepts it too.

A path filter narrows what is *reported*, not what is parsed, so a filtered run detects the same
orphaned references as a full run. Failures are scoped by the spec path they belong to, so a
filtered run reports only those inside the filter — except a malformed reference, which names no
spec path and is therefore always reported.

### In CI

```yaml
- name: Spec check
  run: uvx spec-trace
```

## Configuration

Config is optional. Zero config works for a Python or TypeScript repo keeping specs in
`docs/specs`. Settings live in `[tool.spec-trace]` in `pyproject.toml`, or in `spec-trace.toml`
for non-Python repos.

```toml
[tool.spec-trace]
specs_dir = "docs/specs"
max_behaviour_length = 100
exclude_dirs = ["node_modules", ".venv", "dist", "build", "__pycache__", ".git"]
exclude_files = ["tests/fixtures_test.py"]   # files whose SPEC: text is data, not a reference

[[tool.spec-trace.languages]]
name = "php"
test_glob = "*Test.php"
comment = "//"
test_function = 'function\s+(test\w+)'

[tool.spec-trace.test_types]   # optional; path prefix -> label in coverage output
"integration-tests/" = "e2e"
```

Every list setting **replaces** its default rather than adding to it. Setting `exclude_dirs` drops
the built-in exclusions, so repeat the ones you still want. Defining any language likewise replaces
both built-in languages.

`exclude_files` skips a matching test file entirely, so a real `SPEC:` reference inside it is
ignored too. Reach for it only when a file contains reference-shaped text as data — a better fix is
usually to build that text from parts, as `tests/spec_fixtures.py` does here.

## For coding agents

An agent meets this tool at three moments, and each has its own surface:

1. **While planning** — `init` installs repo instructions (a Claude Code skill plus an `AGENTS.md`
   section) so specs get written *with* the work, not patched in after a gate failure.
2. **On first contact** — `--help` carries a compressed convention: both formats, one example of
   each, and what the gate fails on.
3. **When the gate fails** — every ERROR states the fix, and the FAIL line points at
   `spec-trace docs`.

The projected files are deliberately thin: the stable format cheat-sheet inline, everything else
deferred to `spec-trace docs`, whose content ships inside the package and versions with it.
`init --update` rewrites the skill, and rewrites the `AGENTS.md` region between marker comments,
leaving your own text around it alone. It never touches the specs README or a starter spec written
by `--example`: `init` asks you to describe your app in one and to replace the other, so both are
yours after the first write.

## Known limits

Being honest about what this does not do:

- **It checks linkage, not truth.** Nothing verifies that a test actually exercises the behaviour
  its bullet describes. That is delegated to review and agent discipline.
- **The test→spec direction is weak at scale.** A large codebase adopting this will have thousands
  of unspecified tests, a number nobody acts on. Treat it as a backlog signal, not a target.
- **Parsing is regex, not AST.** The "SPEC comment attaches to the next test function" rule is a
  loose heuristic, and TypeScript `describe` nesting is discarded. Acceptable for a lint.
- **A green gate is not coverage.** A spec point counts as tested when any test cites its keycode.
  Nothing checks that the test exercises the behaviour, or that it asserts anything at all. A
  repo with every spec point cited and every assertion deleted still passes. Read a green run as
  "the links resolve", not "the behaviour works" — including for this repo's own gate.

Non-goals: no semantic verification, no test running, no coverage measurement, no MCP server, and
no runtime dependencies.

## Development

```bash
cp .env.example .env           # only needed for a release, but see below
uv sync --extra dev --locked   # run `uv lock` if this reports a stale uv.lock
bin/check                      # every gate CI runs; reports all failures, not just the first
bin/check --only lint          # one gate by name; --list names them all
```

Copy `.env.example` even when you are not releasing: a shell that sets `UV_ENV_FILE=.env` fails
every `uv run` with `No environment file found` until the file exists. An empty token is fine for
the gates.

`bin/check` is the single definition of the gate commands: it is what a contributor runs before
committing, what `bin/publish` runs before releasing, and what CI runs one gate per job. That is
why no gate command is written out below — the table names each gate, not its command.

| Gate | Checks | Local | CI job |
| --- | --- | --- | --- |
| format | Formatting matches `ruff format`. Run `uv run ruff format .` to rewrite. | `bin/check --only format` | `gate (format)` |
| lint | Ruff lint rules, including unused imports. | `bin/check --only lint` | `gate (lint)` |
| typecheck | Pyright over `src/` and `tests/`, basic mode. | `bin/check --only typecheck` | `gate (typecheck)` |
| deadcode | Vulture finds module-level symbols nothing references. | `bin/check --only deadcode` | `gate (deadcode)` |
| test | The pytest suite. | `bin/check --only test` | `test (3.11)`, `test (3.13)` |
| spec-check | The tool gates its own specs, with `--warn` for coverage drift. | `bin/check --only spec-check` | `gate (spec-check)` |

**Tests run on 3.11 and 3.13 in CI; `bin/check` runs them once, on your interpreter.** A
contributor on 3.12 who is green locally can still be red on 3.11, so a passing `bin/check` is
necessary but not sufficient. The other five gates are single-interpreter and match CI exactly.

`CONTRIBUTING.md` covers the rest, including the house rule a drive-by contributor cannot guess:
this repo gates its own specs, so a pull request adding a test with no `SPEC:` reference fails CI.
Changes worth a release note go in the `Unreleased` section of `CHANGELOG.md`.

`spec-trace` uses its own convention: see `docs/specs/`, with the cross-references in
`tests/test_spec_linked.py`, and behaviour below the level of a spec point covered in
`tests/test_internals.py`.

### Release

```bash
bin/publish [--minor|--major] [--dry-run]
```

That is the whole release. It rebases onto `origin/main`, runs `bin/check`, writes the new
version, retitles the changelog, commits, builds, publishes to PyPI, tags, and pushes the commit
and the tag. Needs `UV_PUBLISH_TOKEN` in the environment or in `.env`, a clean working tree, a
non-empty `Unreleased` section in `CHANGELOG.md`, and `mael` on `PATH` for the rebase.

`--dry-run` runs the gates and builds without publishing, tagging or rebasing, then reverts the
version bump. A failed upload rolls the release commit back, so a re-run starts from the same
version number rather than skipping one.

## Related

[maelstrom](https://github.com/sminnee/maelstrom) is a sibling tool for orchestrating parallel
agent development. The two share a distribution pattern but have no dependency in either
direction: spec-trace is repo-scoped and must run in CI with nothing but Python.

## Licence

MIT
