Metadata-Version: 2.4
Name: whyfile
Version: 0.3.1
Summary: Whyfile: intent-driven development for your codebase. Extract the decisions, constraints, and trade-offs behind your code as grounded, anchored graph nodes, and query them back.
Project-URL: Homepage, https://github.com/whyfile/whyfile
Project-URL: Repository, https://github.com/whyfile/whyfile
Project-URL: Issues, https://github.com/whyfile/whyfile/issues
Project-URL: Changelog, https://github.com/whyfile/whyfile/blob/main/CHANGELOG.md
Author-email: Will Neill <willneill@gmail.com>
License: MIT
License-File: LICENSE
Keywords: claude,claude-code,code-intelligence,documentation,embeddings,graphify,intent,knowledge-graph,llm,rag,static-analysis
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Documentation
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Topic :: Software Development :: Documentation
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: graphifyy>=0.9.3
Requires-Dist: rapidfuzz>=3
Provides-Extra: anthropic
Requires-Dist: graphifyy[anthropic]; extra == 'anthropic'
Provides-Extra: dev
Requires-Dist: mcp<2,>=1.19; extra == 'dev'
Requires-Dist: mypy>=1.11; extra == 'dev'
Requires-Dist: networkx>=3; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest-cov>=5; extra == 'dev'
Requires-Dist: pytest>=7; extra == 'dev'
Requires-Dist: ruff>=0.6; extra == 'dev'
Provides-Extra: embeddings
Requires-Dist: model2vec>=0.3; extra == 'embeddings'
Provides-Extra: gemini
Requires-Dist: graphifyy[gemini]; extra == 'gemini'
Provides-Extra: mcp
Requires-Dist: mcp<2,>=1.19; extra == 'mcp'
Description-Content-Type: text/markdown

# Whyfile

[![PyPI version](https://img.shields.io/pypi/v/whyfile.svg)](https://pypi.org/project/whyfile/)
[![Python versions](https://img.shields.io/pypi/pyversions/whyfile.svg)](https://pypi.org/project/whyfile/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![CI](https://github.com/whyfile/whyfile/actions/workflows/ci.yml/badge.svg)](https://github.com/whyfile/whyfile/actions/workflows/ci.yml)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)

Your codebase remembers **what** it does. It forgot **why.**

`whyfile` reconstructs that why, keeps it honest — every unit tagged `authored` or `reconstructed`, so a model's guess never poses as a decision you made — and puts it on a gate that fails the PR quietly breaking it. Every repo has a Makefile; this is the missing Whyfile. (Formerly `graphify-intent`; see ADR-0024.)

## Why "Whyfile"?

Every repo has a Makefile that says how to build, a Dockerfile that says how to run, and a lockfile that says exactly what ships. None of them say why. Whyfile is the missing file: the decisions, constraints, and trade-offs behind your code, extracted into a queryable graph, anchored to the code they govern, and kept honest by provenance tiers that separate what a human recorded from what a model reconstructed.

The name is also the roadmap. Intent-driven development has one invariant: no unexplained change. Every change either conforms to recorded intent, supersedes it explicitly, or records a new decision. The conformance gate (`whyfile intent-diff --gate`) enforces it: a pull request that silently violates a recorded constraint does not merge. The why goes on file, and the file bears load.

## What it does

`whyfile` reconstructs the *why* your code lost: the decisions, the dead ends, and the constraints the awkward bits are quietly paying for — the reasoning that never became a comment because nobody writes `# WHY:` above the thing they decided *not* to do. Surfacing rationale that's already written down is table stakes; two things make *reconstructed* intent trustworthy instead of plausible-sounding fiction:

- **It's kept honest by provenance.** Every unit is tagged `authored` (a human recorded it) or `reconstructed` (a model inferred it), grounded to the exact source span it came from, so a model's guess never outranks the record or poses as a decision you made.
- **It's load-bearing, not decorative.** `whyfile intent-diff --gate` fails a pull request that changes constraint-governed code without conforming to the constraint or explicitly superseding the decision behind it. The why doesn't just sit in a graph you query — it can block your merge.

Under the hood it reads a [graphify](https://github.com/safishamsi/graphify) knowledge graph alongside your prose docs, runs a short pipeline of LLM passes (extract → anchor → cross-doc relate, plus an opt-in concept-resolution pass), and writes a sidecar JSON, an enriched `graph.json`, and a report. Three LLM-free query commands read it back. Think of it as the month of code archaeology every new hire does on their way in — done once, checked in, and enforced.

```mermaid
flowchart LR
    D["docs/*.md"] --> A
    G["graph.json"] --> B
    A["Pass A<br/>extract intent"] --> B["Pass B<br/>anchor to concepts"]
    B --> C["Pass C<br/>cross-doc intent"]
    B --> R["Pass D<br/>concept resolution<br/>(opt-in)"]
    A --> OUT
    C --> OUT
    R --> OUT
    OUT["outputs:<br/>.whyfile.json<br/>graph.enriched.json<br/>enrichment_report.md"]
```

## Requirements

`whyfile` post-processes graphify graphs and depends on `graphifyy` (the `graphify` runtime that every backend calls), which is installed automatically (ADR-0010). By default it uses your **Claude Pro/Max subscription** via the Claude Code CLI, so **no API key is required**. To use a billed API key instead, see [LLM backend](#llm-backend).

## Installation

```bash
pip install whyfile          # brings graphifyy (the graphify runtime) with it
# or as a standalone tool:
uv tool install whyfile
```

Not a Python project? `whyfile` is a standalone dev tool — install it the way you install your other CLIs:

| You have… | Install with |
| --- | --- |
| `uv` | `uvx whyfile` — or `uv tool install whyfile` to put it on PATH |
| `npm` / Node | `npx whyfile` — delegates to `uv`; if `uv` is missing it prints the one-line install |
| neither / no Python | install `uv` (`curl -LsSf https://astral.sh/uv/install.sh \| sh`, `brew install uv`, or `winget install astral-sh.uv`), then `uvx whyfile` — uv brings its own Python |

The LLM backend still needs the `claude` CLI (subscription default) or an API key — see [LLM backend](#llm-backend).

For an **API** backend, add the provider extra so graphify has its SDK (the subscription backend needs neither):

```bash
pip install 'whyfile[anthropic]'   # Anthropic API
pip install 'whyfile[gemini]'      # Gemini API
```

To enable Pass D's embedding-based concept resolution, add the optional extra:

```bash
pip install whyfile[embeddings]   # local model2vec embedder for Pass D
```

Without it, Pass D still runs, degrading gracefully to a lexical-similarity fallback.

## Quick start

```bash
# Uses your Claude Pro/Max subscription by default (needs the `claude` CLI).
whyfile \
  --graph graphify-out/graph.json \
  --docs docs/ \
  --passes A,B,C
```

## See it on this repo

We point the tool at itself, because it's the most honest demo we can give you. A full run over
this project's own [`docs/adr/`](docs/adr/) trail grows 172 code concepts into 234 nodes and pulls
**62 intent units** (18 decisions, 18 mechanisms, 16 constraints, 10 trade-offs) out of the prose,
each grounded to the exact span it came from (**100% grounding**), 53 anchored back to the code
they explain.

**Then the why bears load.** Open a PR that edits `cache.py` — code governed by four constraints
recorded in [ADR-0015](docs/adr/0015-cache-resume.md) — without accounting for them, and the gate
stops the merge:

```console
$ whyfile intent-diff --files src/whyfile/cache.py --gate
Intent diff: introduces 0, supersedes 0, governed 1, conformance-review 1
  ! src/whyfile/cache.py governed by constraint: Restores extraction, not re-sync, …
Conformance gate: action required — conform to each constraint, or supersede its decision record:
  ✗ src/whyfile/cache.py — Restores extraction, not re-sync (ADR-0015) → conform, or supersede ADR-0015
  ✗ src/whyfile/cache.py — Consistent --docs spelling required (ADR-0015) → conform, or supersede ADR-0015
  ✗ src/whyfile/cache.py — End-of-run sidecar write loses work (ADR-0015) → conform, or supersede ADR-0015
  ✗ src/whyfile/cache.py — Corrupt entry treated as miss (ADR-0015) → conform, or supersede ADR-0015
$ echo $?
5
```

That exit 5 *is* the product. The change conforms to each recorded constraint, or the same PR
supersedes ADR-0015 out loud — silently breaking a decision the code is built on is not a path the
merge leaves open.

**And it stays honest about what it knows.** Ask why the code looks the way it does, and every
answer wears its provenance — `reconstructed` (a model inferred it) never dressed up as `authored`
(a human recorded it):

```console
$ whyfile explain cache.py --format text
cache.py (code)
  • Orphan cache entries accumulate [reconstructed tradeoff] — Orphan cache entries accumulate
    over a corpus's lifetime with no automatic pruning.
    why: Deferring cleanup accepts unbounded cache growth to avoid deleting live entries, because
    auto-prune cannot safely distinguish orphaned entries from ones excluded by the current run.
    alternatives: Auto-prune or an immediate --prune-cache flag, deferred because --docs subsetting
    makes automatic pruning unsafe.
```

That rationale isn't a comment in `cache.py` — it lives in
[ADR-0015](docs/adr/0015-cache-resume.md), which the tool read, distilled, tagged `reconstructed`,
and wired back to the file the decision governs. Surfacing that round trip is table stakes.
**Gating on it — and never letting a model's guess pose as ground truth — is the product.** (The
query commands are covered in [Querying the intent layer](#querying-the-intent-layer) below.)

## LLM backend

By default, `whyfile` uses your **Claude Pro/Max subscription** through the
[Claude Code CLI](https://docs.claude.com/en/docs/claude-code) (`claude`): no API key,
no per-token billing (the run still counts against your plan's usage limits; see
[What a run costs](#what-a-run-costs)). It falls back to a billed API key if the CLI isn't
available.

| `--backend` | Behaviour |
|-------------|-----------|
| *(unset)* | **Subscription** if the `claude` CLI is on your `PATH`, else a detected API key |
| `subscription` | Force the Claude Code CLI (subscription auth) |
| `api` | Force a billed API key (`ANTHROPIC_API_KEY` / `GEMINI_API_KEY`) |
| `claude` / `gemini` / … | Force a specific graphify backend |

You can also set `GRAPHIFY_INTENT_BACKEND`. For subscription mode, install the `claude`
CLI and run it once to sign in. For API mode, install the matching extra
(`graphifyy[anthropic]` or `graphifyy[gemini]`) and provide the key.

### Keep API keys out of `.env` and your shell history

If you do use an API key, prefer a password-manager CLI over inlining the secret. With
[1Password's `op`](https://developer.1password.com/docs/cli/), inject it at runtime so it
never lands on disk in plaintext:

```bash
export ANTHROPIC_API_KEY="$(op read 'op://<vault>/<item>/credential')"
whyfile --backend api --graph graphify-out/graph.json --docs docs/
```

…or wrap the command with `op run` so the secret lives only for that process:

```bash
op run --env-file=.env.op -- whyfile --backend api --graph graphify-out/graph.json --docs docs/
```

A real key committed in `.env` risks leaking into git history, CI logs, and backups; a
manager keeps it encrypted, access-audited, and revocable. Best of all, the default
subscription backend needs no key at all.

## CLI flags

| Flag | Default | Description |
|------|---------|-------------|
| `--graph` | `graphify-out/graph.json` | Path to the graphify output `graph.json` |
| `--docs` | `docs/` | Directory containing markdown source files |
| `--passes` | `A,B,C` | Comma-separated passes: `A` extract, `B` anchor, `C` cross-doc relate, `D` cross-doc concept resolution (opt-in) |
| `--backend` | *(auto)* | Backend selection (see [LLM backend](#llm-backend)) |
| `--progress` | `auto` | Progress reporting mode: `auto` (live line on TTY, heartbeat when piped), `plain` (heartbeat lines only), `json` (NDJSON events on stderr), `none` (silent). stdout stays results only; warnings and progress go to stderr. |
| `--min-confidence` | `0.75` | Minimum `confidence_score` for Pass C **and Pass D** edges |
| `--max-concurrency` | `4` | Maximum number of concurrent LLM requests |
| `--max-tokens` | `500000` | Estimated-token ceiling for the **whole run** (all passes, summed). Enforced three ways (a pre-flight prompt, a mid-Pass-A breaker, and a post-Pass-A checkpoint before Pass B), so a run stays within the estimate it showed you. Nothing is written on an abort; `--yes` skips only the pre-flight prompt, never the checkpoint. `0` = unlimited. Full semantics in [`AGENTS.md`](AGENTS.md) |
| `--max-cost-usd` | `2.00` | Cost guard for **API** backends: prompts if the pre-flight estimate exceeds this (the subscription backend has no per-token bill to guard, so it skips this check; `--max-tokens` is the guard that applies there). Pre-flight only; the post-Pass-A checkpoint gates tokens, not cost |
| `--yes` | `false` | Skip the pre-flight ceiling prompt (non-interactive / CI); does *not* skip the post-Pass-A whole-run checkpoint |
| `--dry-run` | `false` | Project calls/tokens/cost/time as JSON and exit: no extraction (one tiny backend liveness ping only). JSON shape documented in [`AGENTS.md`](AGENTS.md) |
| `--no-cache` | `false` | Bypass the Pass A cache entirely: no reads, no writes; every section re-extracts. See [Caching & resume](#caching--resume) |
| `--doctor [--json]` | `false` | Check the environment (runtime, backend auth, graph/docs shape, version drift) and exit; `--json` for machine output. Version drift is advisory only; a stale editable install no longer fails the run |
| `--version` | n/a | Print the installed version and exit |

Set `--backend` to `subscription`, `api`, or an explicit provider name (see [LLM backend](#llm-backend)).

**stdout is strictly results-only.** The pre-flight projection line (`Pass A: ~… tokens
(whole run) …`), the `Pass A: gated N section(s)` prefilter notice, and every
ceiling/prompt/abort message print to stderr; stdout carries only the per-pass result lines,
the sidecar path, and `Done.`.

### Exit codes

Stable across releases (`src/whyfile/errors.py`), safe for scripts/agents to branch on:

| Code | Meaning |
|------|---------|
| `0` | Success |
| `1` | Unexpected error (unhandled exception; never raised deliberately) |
| `2` | Bad input: invalid `--passes`, `graph.json` missing, no `.md` docs found |
| `3` | Token/cost ceiling: pre-flight decline, non-interactive over-ceiling, the Pass A budget breaker, or the post-Pass-A whole-run checkpoint before Pass B |
| `4` | Backend unusable: none configured, the graphify runtime missing, or all Pass A sections failed |
| `5` | Policy gate: `changed --fail-on-constraint` (a changed file is governed by a constraint), `intent-diff --gate` (constraint-governed code changed with no superseding record in the PR), or `check` (a compiled constraint in `intent-rules.json` is violated) |

See [`AGENTS.md`](AGENTS.md) for the full contract (per-code agent actions, `--dry-run`/`--doctor`
JSON shapes, progress events, and manifest fields) for driving this tool non-interactively.

### Per-pass models (subscription backend)

On the subscription (`claude-cli`) backend, each pass defaults to a different model rather
than one blanket choice, which kills the old Opus-for-everything default:

| Pass | Default model |
|------|---------------|
| A (extract) | `sonnet` |
| B (anchor) | `haiku` |
| C (cross-doc relate) | `haiku` |
| D (concept resolution) | `haiku` |

Pass A keeps the stronger model because it does the actual reasoning (deriving a claim,
rationale, and alternatives from prose); B/C/D are comparatively mechanical, so `haiku`
carries them at a fraction of the cost. Set `GRAPHIFY_CLAUDE_CLI_MODEL` to override the
model for **every** pass, e.g. `GRAPHIFY_CLAUDE_CLI_MODEL=haiku` if your corpus is
code-heavy enough that even Pass A doesn't need `sonnet`-level reasoning.

**`--max-concurrency` changes wall-clock time only, not cost, and only on the API backend.**
The actual cost levers are the model (per-pass tiering above), the call count (Pass A's
structural gate on trivial sections), and per-call context; raising concurrency just runs the
same calls faster in parallel, there. On the subscription backend, claude-cli calls serialize
upstream in graphify (a lock around the CLI subprocess), so `--max-concurrency` has little
effect: raising it does not make subscription runs faster.

## Caching & resume

Pass A caches each section's extraction on disk at `graphify-out/.intent_cache/` (next to
`graph.json`), keyed by a content hash of the exact prompt (source file + section path + body),
the resolved Pass A model, and a fingerprint of the system prompt + schema. Each entry is
written the moment its section completes.

- **Automatic reuse.** Re-running the same corpus serves already-extracted sections from
  cache; only new or edited sections (their body hash changed) re-extract. The pre-flight
  estimate, ETA, cost gate, `--dry-run` projection, and the post-Pass-A checkpoint all size
  Pass A off the cache-**miss** sections only, so a fully cache-warm re-run projects ~0
  calls/tokens for Pass A.
- **Resume after an abort.** If a run aborts (either the mid-Pass-A token breaker or the
  post-Pass-A `--max-tokens` whole-run checkpoint before Pass B), every section that
  finished before the abort is already cached. Just re-run (after raising `--max-tokens`,
  or narrowing `--docs`) and Pass A resumes from where it left off; the abort message says
  so.
- **`--no-cache`** bypasses the cache completely for the run: no reads, no writes.
- **Force a refresh** by deleting `graphify-out/.intent_cache/`; there's no separate
  "clear cache" flag.
- **Use a consistent `--docs` spelling.** The cache key embeds the source-file path string
  as `--docs` produces it, so re-running against the same docs with a differently-spelled
  `--docs` (e.g. relative vs. absolute) can miss the cache even though the files are the same.

## What a run costs

On an **API** backend you pay for real tokens, and it stays cheap. whyfile scales with your
**docs**, not your lines of code (the base graphify graph for source is AST-based and free), so the
bill tracks how much prose you point it at. Pass A does the reasoning on `sonnet` at roughly **half
a cent per doc section** and dominates the total; Pass B/C/D run on `haiku` for pennies.

On the **default subscription backend** there is no per-token bill: the run goes through your
Claude Pro/Max plan, so there is no separate dollar cost. But the run is not free: it still
consumes the estimated tokens in the table below against your plan's usage limits, the same
tokens an API run would be billed for. "No bill" is not "no cost".

| Corpus | Doc sections | Intent units | Est. tokens | API cost |
|--------|-------------:|-------------:|------------:|---------:|
| This repo's `docs/adr` (measured) | 55 | 62 | 65.6k | ~$0.33 |
| Medium doc set | ~200 | ~225 | 230k | ~$1.18 |
| Large doc set | ~800 | ~900 | 914k | ~$4.71 |

Passes A,B,C (the default); adding Pass D moves the total by under a cent. The first row is measured
on this repository; the other two are the tool's own projections at representative sizes, priced at
the published API rates (`sonnet` $3/$15, `haiku` $0.80/$4.00 per Mtok). These are cold-cache,
from-scratch figures: because Pass A caches every section, a **re-run costs about $0 on API**
(only new or edited sections re-extract); on subscription a re-run still costs about zero
tokens, for the same cache-hit reason.

`--dry-run` and the end-of-run manifest report this honestly: `est_cost_usd` is a real number on
the API backend, but `null` on subscription (never `0.0`, which would read as free), paired with a
`billing` note stating the estimated token count and that it counts against your plan.

To see the exact projection for your own corpus before spending anything, `--dry-run` prints it as
JSON and makes no extraction calls:

```bash
whyfile --dry-run --graph graphify-out/graph.json --docs docs/
```

## Outputs

All outputs are written to the same directory as `graph.json`:

| File | Contents |
|------|----------|
| `.whyfile.json` | Sidecar: all intent nodes and edges added across runs |
| `graph.enriched.json` | Original graph merged with the intent layer (graphify node-link format) |
| `enrichment_report.md` | Before/after metrics: node/link/isolated counts, intent breakdown by kind, anchored vs orphaned, grounding coverage |
| `.whyfile_progress.jsonl` | Progress stream (NDJSON, one event per line): `run_start`, `pass_start`, `tick`, `unit_fail`, `pass_end` events for agents to tail or poll. Written for any `--progress` mode except `none`. |
| `intent_run.json` | Per-run manifest: `status`/`error`, `metrics`, `artifacts`, and per-pass records (`passes[]`, including `s_per_call`, `cache_hits`, and `cache_misses`; the cache fields are nonzero only for Pass A). Written on every run (independent of `--progress` mode). At run end, an estimated-vs-actual runtime/duration reconciliation line is printed to stderr. |
| `intent_runs.jsonl` | History: the same manifest object appended as one line per run, oldest first: a run-over-run log without snapshotting `intent_run.json` yourself. |
| `.whyfile.prev.json` | Snapshot of the sidecar as it was *before* this run's merge. Written every run (skipped on the first run). Powers `digest --since last-run`. Git-ignored under `graphify-out/`. |

## Querying the intent layer

Once a run has produced its outputs, three subcommands read them back: **read-only, LLM-free,
and free of charge** (no backend call, no cost). Each prints exactly one **JSON object on
stdout by default**; add `--format text` for a human-readable render instead. All three accept
`--graph PATH` (default `graphify-out/graph.json`, same as the pipeline; may be given before or
after the subcommand name) and exit `0` for a `status` of `ok`/`no_strong_match`, `2` for
`not_found`/`ambiguous`/`no_intent_layer` (see [Exit codes](#exit-codes); full JSON schemas are
in [`AGENTS.md`](AGENTS.md)).

**`explain <node>`** resolves `<node>` (an exact node id, else a rapidfuzz match over node
labels) and explains it: for a **code/document node**, the intent node(s) that motivate it (the
intent-origin `rationale_for` edges pointing at it); for an **intent node**, its own
claim/rationale, the node(s) it explains, and its Pass-C relations (a supersession is surfaced
as `superseded_by` on the superseded node). Requires `graph.enriched.json`; returns
`status: "no_intent_layer"` (exit `2`) if it doesn't exist yet.

```bash
whyfile explain AuthModule
whyfile explain i_d1 --format text
```

**`list-intent [--kind decision|mechanism|constraint|tradeoff] [--min-confidence F]`** returns the
intent inventory, optionally filtered by kind and/or a minimum `confidence_score`.

```bash
whyfile list-intent --kind decision --min-confidence 0.8
```

**`why "<question>"`** ranks intent nodes against free text with rapidfuzz token matching and
returns the top `--top` (default `5`) by score (0-100); a top score below the cutoff (60) still
returns its results, just with `status: "no_strong_match"` so a caller can tell the match was
weak.

```console
$ whyfile why "why does Pass A use a stronger model than the others" --top 2 --format text
Top 2 match(es) (no_strong_match):
   56  [tradeoff] Per-pass tiering protects Pass A quality — Running Opus on every pass was the
       cost bug; tiering lets cheaper models handle most passes, but Pass A is deliberately held
       on sonnet because its output quality matters enough to justify the higher tier.
   56  [decision] Per-pass model tiering — Passes B/C/D need less reasoning than A, so downgrading
       them to cheaper haiku captures most of the cost saving without hurting quality — but a
       user's explicit model choice still wins.
```

`list-intent` and `why` don't require `graph.enriched.json`; they fall back to the
`.whyfile.json` sidecar when it's the only artifact present. The pipeline itself is
unchanged: running `whyfile` with no subcommand still runs the extract/anchor/relate
run described above.

## Team collaboration

whyfile turns the tribal "why is the code like this?" into a layer that is queryable,
PR-reviewable, and survives turnover. Three jobs a team gets for free (all LLM-free, $0 at query
time):

- **Onboard.** A new hire runs `whyfile explain <file>` instead of interrupting a senior.
- **Review.** `whyfile changed --base main` surfaces the intent a PR touches, so a
  reviewer sees the constraints a change is bound by.
- **Retain.** `whyfile coverage` shows which code has recorded intent and which is dark,
  and `whyfile digest` reports what was decided since a checkpoint.

### The PR check

```bash
git diff --name-only origin/main...HEAD | whyfile changed --format markdown
```

Maps each changed file to the intent that governs it (constraints and trade-offs first). It is
informational by default (exit 0). Add `--fail-on-constraint` to turn it into a soft gate that
exits 5 when a PR touches a file bound by a constraint, so the constraint gets a reviewer's
acknowledgement.

### Coverage and digest

```bash
whyfile coverage --format text       # golden fraction, intent debt, covered vs dark code
whyfile digest --since last-run       # what intent changed since the last run
```

`coverage` reports the **golden fraction** beside the coverage numbers: the share of intent that is
trusted evidence (human-written: authored decision records, captured live decisions, and attested
commit trailers / `# why:` comments; the numerator grows further as the confirmed tier lands) over total
intent. It is partitioned by evidence class and never blended, so reconstructed intent is shown
separately and never inflates the trusted count. Alongside it, `intent_debt` reports dark files and
orphaned intent (intent that explains no code); stale decisions come from `drift` and disputes from
the audit phase, reported as `null` here rather than a misleading zero.

The **attested tier** is the zero-workflow capture channel: on every pipeline run (unless
`--no-attest`), commit trailers (`Why:`/`Decision:`) and `# why:` code comments are ingested
deterministically as `provenance: attested` intent nodes, anchored to the code they touch or
annotate. This gives any repo a non-empty trusted layer on day one, with no new workflow to adopt.
List them with `whyfile list-intent --provenance attested`.

The **capture** subcommand records a decision at the moment of choosing, the cheapest time to
capture ground truth:

```bash
whyfile capture --chosen "Use SQLite" --question "Which datastore?" \
  --option "Use SQLite" --option "Use Postgres" --rationale "single-writer, zero-ops" \
  --recommendation "Use SQLite"
```

It writes a `# Decision:` record to `docs/decisions/` (idempotent by slug) and merges a
`provenance: captured` node into the graph, immediately queryable via `why`/`explain`/`list-intent
--provenance captured`. The node is built through the same ingest path a committed record takes, so
re-ingesting the record reproduces the identical node. If `--recommendation` differs from
`--chosen`, the node records a `resolution_delta` capturing where the human overrode the agent.

`--since last-run` needs at least two pipeline runs: the first run has no prior snapshot to diff
against, so it writes none and `digest --since last-run` exits `2` with a clear message until a
second run has happened.

### Drift and affirmation

Recorded decisions rot silently when the code they explain changes underneath them. `affirm` stamps
a decision as reviewed against the current code; `drift` flags any affirmed decision whose anchored
code has changed since, so staleness is visible instead of silent.

```bash
whyfile affirm <decision>     # "I reviewed this against the code; it still holds"
whyfile drift --format text   # decisions whose anchored code changed since affirming
```

`affirm` records a content fingerprint of the anchored code (symbol-level where the anchor names a
function or class, file-level otherwise) into `intent-affirmations.json`, a small human-owned ledger
you commit. `drift` recomputes those fingerprints and reports four buckets: needs-reaffirmation
(affirmed, but the code moved), expired-assumptions (see below), never-affirmed (anchored, staleness
unknown), and orphaned (a ledger entry for a decision no longer in the graph). It is deterministic,
LLM-free, and informational (exit 0); re-affirming a flagged decision clears it.

Assumptions are the other decay mechanism: most stale intent is stale because an assumption quietly
expired. A record's `## Assumptions` section (see `docs/adr/TEMPLATE.md`) is ingested into authored
`assumption` intent nodes, each with an optional review-by date:

```bash
whyfile list-intent --kind assumption      # the facts your decisions bet on
whyfile drift --as-of 2026-12-31           # what is overdue for review by that date
```

An assumption past its review-by date surfaces in `drift` as an expired assumption, prompting you to
revisit the decision that rests on it. `--as-of` defaults to today; pass a future date to see what
will be overdue by then. Because assumptions are first-class intent nodes, `explain <file>` and
`why "..."` also answer "which assumption governs this path?" during incident triage.

### Trust lifecycle

Trust is a lifecycle, not a label. `trust` resolves each intent node's current state: it rises
through `provisional` (reconstructed, unverified), `corroborated`, `confirmed` (survived an audit),
and `affirmed` (a human signed off), and falls to `stale` (affirmed but the code drifted since),
`disputed`, `superseded` (a newer record replaced it), or `orphaned` (its anchored code was
deleted).

```bash
whyfile trust --format text                          # every node's state, counted by state
whyfile trust <node> --set confirmed --by panel      # record a court verdict (who/what/when)
```

States are derived deterministically from the graph plus the affirmation ledger plus a recorded
verdict log (`intent-trust.json`), so they are stable across re-runs. Structural facts (superseded,
orphaned) win over promotions, and a human `affirm` outranks a panel verdict. Recording a verdict
appends to the log with the actor and timestamp; it never edits the graph. Superseded and orphaned
states are detected automatically from the graph; confirmed and disputed come from recorded verdicts
today, and from the audit panel once it lands.

### Ongoing cost per PR

Updating the intent layer on a PR is cheap: cents on an API backend (a ~$0.03 floor), or a small
token draw against your plan's usage limits on the subscription backend, no separate bill either
way. A per-PR update re-anchors existing intent plus any changed doc sections; the Pass A cache
means unchanged sections cost nothing on either backend. See
[What a run costs](#what-a-run-costs) for the full breakdown.

### Two CI postures

| Posture | CI cost | Needs a secret? | Freshness |
|---|---|:--:|---|
| Regenerate in CI | ~cents/PR | Yes (API key) | Always current |
| Committed sidecar | $0 | No | Refreshed by a separate job |

Example workflows for both live in [`examples/workflows/`](examples/workflows/).

### Roadmap

Cached Pass B anchoring (to make per-PR updates truly diff-proportional) and `--since` pipeline
scoping are planned.

## MCP server

The same read-only query layer is available to IDEs and coding agents over the
[Model Context Protocol](https://modelcontextprotocol.io), so the *why* is one tool call away
without shelling out. Install the extra and start a stdio server:

```bash
pip install 'whyfile[mcp]'
whyfile --mcp --graph graphify-out/graph.json
```

Register it with a client (Claude Code, Cursor, IDE agents). For Claude Code, add to `.mcp.json`
at the project root (launch the server from the project root so a `--base`-style git diff resolves
against your repo):

```json
{
  "mcpServers": {
    "whyfile": {
      "command": "whyfile",
      "args": ["--mcp", "--graph", "graphify-out/graph.json"]
    }
  }
}
```

Seven `intent_*` tools are exposed: six read-only query tools plus one explicit write tool, each a
thin wrapper over the CLI command of the same name, so a tool result is identical to its CLI
counterpart's JSON:

| Tool | Inputs | What it does |
|------|--------|-----------------|
| `intent_explain` | `node` | Why is *this* node like this: the intent that motivates a code/doc node, or an intent node's claim, rationale, alternatives, and relations. |
| `intent_list` | `kind?`, `min_confidence?`, `provenance?` | The intent inventory, optionally filtered. |
| `intent_why` | `question`, `top?` | Ranked intent matching a natural-language question. |
| `intent_changed` | `files[]` or `base` | The intent governing changed files, constraints first. |
| `intent_coverage` | (none) | Which code has recorded intent and which is dark. |
| `intent_digest` | `since` | What intent was added, removed, or superseded since a checkpoint. |
| `intent_capture` (**write**) | `chosen`, `rationale`, `question?`, `options?`, `recommendation?` | Persist a resolved decision as a record + a `captured` node, without shelling out. |

The six query tools are **read-only, LLM-free, and free at query time** (they read the local graph,
make no backend call). `intent_capture` is the one **write** tool (B5): it is explicit (a distinctly
named tool, never a mode on a read tool), schema-validated, and scoped to capturing a decision, so
the read-only guarantee of the others is unchanged. Results carry both a JSON body and MCP
`structuredContent`; a real failure sets
`isError`, while a missing intent layer returns a graceful `no_intent_layer` result rather than an
error. The server only reads a local graph: no mutations, no pipeline runs, no secrets, safe to
hand to any agent. See [ADR-0018](docs/adr/0018-mcp-server.md) for the design.

## How it works

**Pass A, intent extraction.** Each markdown document is split into heading-bounded sections. Per section, the LLM extracts 0-N *intent units* (a `decision`, `mechanism`, `constraint`, or `tradeoff`), each with a one-sentence `claim`, the `rationale` (the *why*), any `alternatives` considered, and a deterministic ID grounded to the section's character span.

**Pass B, anchoring.** Each intent node is linked to the graph concept(s) it explains via a `rationale_for` edge. Candidate graph nodes are found **without embeddings** (same `source_file` plus lexical overlap), then the LLM adjudicates which the intent actually explains. Nodes are adjudicated in small batches (several per call) for throughput, each keeping its own candidate list so batching does not blur anchoring (ADR-0019).

**Pass C, cross-doc intent.** The intent nodes are related in bounded chunks (one LLM call per chunk) to surface intent-level relationships between them: `supersedes`, `trade_off_against`, `constrains`, `motivated_by`, `contradicts`. Edges below `--min-confidence` are discarded. Chunking keeps each call bounded; relationships between nodes in different chunks are not surfaced (ADR-0019).

**Pass D, cross-doc concept resolution (opt-in).** Enable with `--passes A,B,C,D`.
Pass D unifies the graphify concepts the intent layer touches **non-destructively**,
emitting `same_concept` edges between existing concept nodes (nothing is merged or
rewritten). Cross-document intent linkage then emerges transitively:
`intent -[rationale_for]-> JWT -[same_concept]- token <-[rationale_for]- intent`. Candidate
pairs are cross-file only, blocked by **embedding cosine** when a local model is
available (optional `[embeddings]` extra) or **lexical Jaccard** otherwise, then the LLM
adjudicates each pair; edges below `--min-confidence` are dropped. Every edge records its
`method` (embedding/lexical) and `similarity` for provenance.

## Idempotency

Re-running `whyfile` is safe. Intent nodes already present in the sidecar (`.whyfile.json`) are skipped by ID, and edges are skipped by `(source, target, relation)` key, so a re-run adds only genuinely new intent. You can also run passes separately (e.g. `--passes A` then `--passes B,C`): when `A` is not in the run, later passes seed from the existing sidecar.

## Development

```bash
git clone <repo>
cd whyfile
pip install -e ".[dev]"
python -m pytest tests/ -v
```

Tests cover every module: IDs, section splitting with spans, relation/confidence validation, all four passes (including Pass D's candidate resolution and embedding fallback), merge/enriched-graph assembly, the report, backend resolution, and an end-to-end smoke test. The LLM boundary and the graphify-runtime probe are injected/mocked, so the suite makes no network calls and needs no API key or live backend.

## graphify runtime dependency

Every backend routes through graphify's `graphify.llm`, so `whyfile` declares `graphifyy` as a **runtime dependency**; `pip install whyfile` installs it (ADR-0010). For an API backend, add the provider extra so graphify's SDK is present:

```bash
pip install 'whyfile[anthropic]'   # --backend api with Anthropic
pip install 'whyfile[gemini]'      # --backend api with Gemini
```

The default subscription backend needs no provider SDK, only the [Claude Code CLI](https://docs.claude.com/en/docs/claude-code) (`claude`), installed and signed in. If the graphify runtime is somehow missing, `whyfile` exits immediately with a clear install message rather than failing deep inside a run.

## Building the plane while we fly it

Structure tools map imports and call graphs: the *what* and the *how-it-connects*. What they
can't hand you is the reasoning: which decision a file is the consequence of, what got tried and
thrown away, why the ugly workaround earns its keep. `whyfile` is a thin *reasoning* layer
over the structural graph graphify already builds: a decision/strategy view, not another entity
extractor, and deliberately **not** a vector DB or a RAG stack (that would be a much heavier
machine than the job needs).

The [See it on this repo](#see-it-on-this-repo) demo up top wasn't a one-off. This whole
[`docs/adr/`](docs/adr/) trail is fair game (decisions, rationale, and the alternatives
considered), except we wrote them by hand *as we built it*, so you can watch the reasoning
actually move:

- **[ADR-0002](docs/adr/0002-intent-rationale-wedge-and-data-model.md)** kills the
  predecessor's generic "concept extraction" and bets the whole tool on intent: the one
  niche the graph *constructors* leave empty.
- **[ADR-0004](docs/adr/0004-embedding-free-same-file-anchoring.md)** ships v1 deliberately
  embedding-free and *defers* cross-document resolution… which
  **[ADR-0009](docs/adr/0009-cross-document-concept-resolution.md)** then delivers, its design
  shaped by *reading graphify's own code* and discovering its dedup engine couldn't do the job.
- **[ADR-0010](docs/adr/0010-graphify-hard-runtime-dependency.md)** is the fun one: it openly
  *supersedes an earlier README claim* after a clean install exposed a silently broken package.
  The old rationale, in its own words, "does not survive contact with the code."
  **[ADR-0011](docs/adr/0011-cheaper-intent-passes.md)** then catches extraction quietly running
  on Opus ("overkill," per the code's own comment) and puts it on a budget.

That last beat, *a decision that doesn't survive contact with the code*, recurs across three
ADRs, and it's the whole reason the tool exists: it's the reasoning that lives in a maintainer's
head and evaporates the day they leave. We're building the plane while flying it; the flight
recorder is checked in right below.

## Architecture decisions

Design rationale is recorded as ADRs in [`docs/adr/`](docs/adr/):

| ADR | Decision |
|-----|----------|
| [0001](docs/adr/0001-direct-schema-enforced-llm-calls.md) | Direct, schema-enforced LLM calls |
| [0002](docs/adr/0002-intent-rationale-wedge-and-data-model.md) | Intent/rationale as the product wedge + node data model |
| [0003](docs/adr/0003-span-level-grounding-and-provenance.md) | Span-level grounding & provenance |
| [0004](docs/adr/0004-embedding-free-same-file-anchoring.md) | Embedding-free, same-file anchoring (v1) |
| [0005](docs/adr/0005-extend-graphify-relation-vocabulary.md) | Extend graphify's relation vocabulary deliberately |
| [0006](docs/adr/0006-enriched-graph-assembly-node-link.md) | Enriched-graph assembly on the node-link format |
| [0007](docs/adr/0007-determinism-idempotency-confidence.md) | Determinism, idempotency, and a single confidence policy |
| [0008](docs/adr/0008-subscription-first-llm-backend.md) | Subscription-first LLM backend via the Claude CLI |
| [0009](docs/adr/0009-cross-document-concept-resolution.md) | Cross-document concept resolution (Pass D) |
| [0010](docs/adr/0010-graphify-hard-runtime-dependency.md) | graphify (`graphifyy`) is a hard runtime dependency |
| [0011](docs/adr/0011-cheaper-intent-passes.md) | Cheaper, safer intent passes (model tiering, gate, estimate, breaker) |
| [0012](docs/adr/0012-progress-observability.md) | Intent-layer progress observability |
| [0013](docs/adr/0013-dx-hardening.md) | Fail-closed automation surface and first-hour DX |
| [0014](docs/adr/0014-whole-run-budget.md) | Whole-run token ceiling + calibrated ETA |
| [0015](docs/adr/0015-cache-resume.md) | Per-section Pass A cache + resume |
| [0016](docs/adr/0016-intent-query-layer.md) | Intent query layer (`explain`/`list-intent`/`why`) |
| [0017](docs/adr/0017-team-collaboration-bundle.md) | Team collaboration bundle (`changed`/`coverage`/`digest` + PR check) |
| [0018](docs/adr/0018-mcp-server.md) | MCP server (`--mcp`) exposing the query layer to IDEs and agents |
| [0019](docs/adr/0019-pipeline-throughput-and-honest-cost.md) | Pipeline throughput (batched Pass B, chunked Pass C, serial-aware estimate) + honest token-based cost reporting |
| [0020](docs/adr/0020-adopt-decision-layer-arc.md) | Adopt the decision-layer arc (capture, ground, audit) |

## License and attribution

`whyfile` is licensed under the [MIT License](LICENSE), Copyright (c) 2026 Will Neill.

This project is an independent post-processor built to interoperate with
[graphify](https://github.com/safishamsi/graphify) by Safi Shamsi. It reuses graphify's
graph schema and relation vocabulary and calls graphify as a separately-installed runtime
dependency; no graphify source code is bundled with or distributed as part of this project.
graphify is licensed under the MIT License (Copyright (c) 2026 Safi Shamsi); see the
`ACKNOWLEDGEMENT AND ATTRIBUTION` section of this repository's [LICENSE](LICENSE) file and
the [upstream license](https://github.com/safishamsi/graphify/blob/v8/LICENSE) for the
full text. With thanks to the graphify project.
