Metadata-Version: 2.4
Name: llmlint-cli
Version: 0.3.8
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Rust
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Dist: oneharness-cli>=0.3.0
License-File: LICENSE
Summary: LLM-as-judge linter: enforce code-quality checks deterministic linters can't express, by driving real coding harnesses through oneharness.
Keywords: cli,linter,llm,code-review,oneharness
Home-Page: https://github.com/nickderobertis/llmlint
Requires-Python: >=3.8
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Changelog, https://github.com/nickderobertis/llmlint/blob/main/CHANGELOG.md
Project-URL: Repository, https://github.com/nickderobertis/llmlint

# llmlint

![llmlint's live progress view: rules resolving one by one as their judges return, then clearing to reveal the report](docs/screenshots/demo.gif)

**The next generation of linting: an LLM as a judge.** `llmlint` enforces the
code-quality checks a human reviewer normally makes — adherence to architectural
patterns, coding-style intent, alignment to organization objectives — that
deterministic linters can't express. It is **additive** to your existing linters,
not a replacement: keep using deterministic tools for everything they can already
check, and reach for llmlint only for the judgment calls.

Each check is a **rule**: a statement about your code that is judged `true`
(holds) or `false` (a violation). A single fast Rust binary, llmlint **batches**
your rules into as few harness calls as it can, then drives a real coding harness
(Claude Code, Codex, Cursor, …) through
[`oneharness`](https://github.com/nickderobertis/oneharness) to read the relevant
files and decide, and reports the violations — with file and line numbers where
they can be pinned down. Because the gate is "just a config file," llmlint drops
into CI next to your other linters.

By default llmlint reports the failing rules (with the locations it could pin
down) and a one-line summary — passing, skipped, and not-relevant rules are just
counted:

![llmlint's default report: a red FAIL with its pinned violation and a colorized summary line counting passed, failed, skipped, and not-relevant rules](docs/screenshots/lint-default.svg)

Add `-v` to itemize *every* rule (passed, skipped, and not relevant too) and to
print the oneharness debug view — the exact `oneharness run …` command and the
raw result for each judge — to **stderr**, so the report on stdout stays clean:

![llmlint's verbose report: green PASS, red FAIL, yellow SKIP, and dim N/A not-relevant lines above the summary](docs/screenshots/lint-verbose.svg)

<details>
<summary>The <code>-v</code> debug view (oneharness command + raw result per judge, on stderr)</summary>

![llmlint's -v oneharness debug view: the exact oneharness run command and the raw JSON result for each judge](docs/screenshots/lint-debug.svg)

</details>

> These are real captures of the CLI, rendered from the actual colorized output
> by [`just screenshots`](screenshots/AGENTS.md) and gated by
> [screencomp](https://github.com/nickderobertis/screencomp).

The exit code is unaffected by verbosity (`0` all-pass, `1` a violation, `2`
the run couldn't complete); operational errors are always shown. Use
`--format json` for the full machine-readable report.

The human report is **colorized** — green `PASS`, red `FAIL`/`ERROR` — when
stdout is a terminal. Coloring follows the [`NO_COLOR`](https://no-color.org)
convention and a `--color <auto|always|never>` flag: `auto` (the default) colors
only an interactive terminal, `always` forces it (e.g. through a pager or to
capture a screenshot), `never` disables it. `--format json` is never colorized.

While the judges run, llmlint draws a **live progress view** on stderr — rules
resolving one by one as their judges return (the GIF above) — then clears it and
prints the report. Like the color, it is audience-aware: `--progress
<auto|always|never>` (default `auto`) shows it only for an interactive human — a
terminal, not CI, and **not an AI coding agent** (Claude Code, Codex, Cursor,
detected via their environment variables). Piped, redirected, in CI, or captured
by an agent, the view is fully suppressed so it never spams a log or an agent's
context — the report on stdout, and `--format json`, are byte-for-byte the same
either way.

## How it works

1. You declare **rules** (and optionally **agents** that group them) in a YAML
   config — like any other linter.
2. For each agent, llmlint renders a system prompt from a template (the rules +
   the target file paths) and calls `oneharness run` with a generated **JSON
   Schema** for structured output. oneharness constrains and validates the
   harness's answer, so llmlint gets a checked verdict per rule, not prose.
3. The harness reads the target files on demand with its own tools to gather
   evidence, then returns `{ "rule_name": { "holds": bool, "violations": [...] } }`.
4. llmlint aggregates (majority vote across judges when configured), reports, and
   exits non-zero if any rule was violated.

llmlint **shells out to oneharness** — it is a runtime prerequisite (see Install).

## Install

`llmlint` needs the `oneharness` binary on your `PATH`.

```console
# One step via pip: prebuilt binaries for llmlint AND oneharness (a dependency)
pip install llmlint-cli

# Or piece by piece:
# 1) oneharness (the harness driver)
curl -fsSL https://raw.githubusercontent.com/nickderobertis/oneharness/main/scripts/install.sh | sh
#    (or: cargo install --git https://github.com/nickderobertis/oneharness --locked)

# 2) llmlint
curl -fsSL https://raw.githubusercontent.com/nickderobertis/llmlint/main/scripts/install.sh | sh
#    (or: cargo install llmlint --locked)
#    (or, without a crates.io release: cargo install --git https://github.com/nickderobertis/llmlint --locked)

llmlint doctor      # confirms oneharness is reachable
```

The installer honors `LLMLINT_VERSION` / `LLMLINT_INSTALL_DIR` (or the `--version`
/ `--to` flags), works on Linux, macOS, and Windows under a POSIX shell
(Git Bash / MSYS / WSL), and refuses a binary it cannot verify. Each tagged
release publishes prebuilt binaries for those platforms, each with a SHA-256
checksum and a keyless [Sigstore](https://www.sigstore.dev/) build-provenance
attestation bundle (`.sigstore.json`); on native Windows PowerShell, use
`cargo install llmlint --locked`.

**Via pip.** Each release also ships per-platform wheels wrapping the same
prebuilt binary, so anywhere Python is present, `pip install llmlint-cli` (or
`uv tool install llmlint-cli` / `pipx install llmlint-cli`) is a seconds-fast
binary install with no Rust toolchain — handy in restricted-egress environments
where package registries are reachable but `github.com` is not. It depends on
[`oneharness-cli`](https://pypi.org/project/oneharness-cli/), so the `oneharness`
runtime prerequisite comes along automatically — one pip install is a complete,
working setup. That includes tool-isolating installers: `uv tool install` and
`pipx` link only llmlint itself onto your `PATH`, but llmlint also looks for
`oneharness` beside its own executable — where those tools place dependency
binaries — so no extra flags are needed. (The PyPI package is `llmlint-cli` — PyPI reserves names too
similar to existing projects — but the installed binary is `llmlint`.) Wheels are published
with PyPI [Trusted Publishing](https://docs.pypi.org/trusted-publishers/) and
carry [PEP 740](https://peps.python.org/pep-0740/) attestations — the same
Sigstore build provenance as the GitHub release assets.

**Behind a mirror.** In a network that can reach a release-proxy mirror but not
`github.com`, point the archive download at it:

```console
LLMLINT_RELEASE_BASE_URL=https://mirror.example/llmlint \
  curl -fsSL https://raw.githubusercontent.com/nickderobertis/llmlint/main/scripts/install.sh | sh
```

The archive comes from the mirror, but its integrity is checked against a trust
root the mirror does not control. If a Sigstore verifier is installed —
[`cosign`](https://github.com/sigstore/cosign), the official
[`sigstore`](https://pypi.org/project/sigstore/) Python client, or
[`gh`](https://cli.github.com/) — the installer downloads the `.sigstore.json`
bundle from the mirror and verifies it **offline** — no GitHub API — against the
keyless signature bound to this repo's release workflow; the trusted digest comes
from the *signed* attestation, so a mirror cannot forge it. With no verifier
installed it falls back to a `.sha256` fetched from canonical GitHub
(`LLMLINT_CHECKSUM_BASE_URL` overrides that root) — but it **refuses** a checksum
that shares the mirror's origin, since a tampered mirror would just serve a
matching tampered checksum. If nothing independent of the mirror can vouch for
the archive, the install aborts.

A verifier is one install away even where `github.com` itself is unreachable,
because all three ship through package registries:

```console
pip install sigstore                                        # PyPI
npm install -g @sigstore/cli                                # npm
go install github.com/sigstore/cosign/v2/cmd/cosign@latest  # Go module proxy
```

(`go install` fetches through `proxy.golang.org` and integrity-checks against the
`sum.golang.org` transparency log — it never contacts github.com.) So the pattern
for a restricted-egress host or sandbox image is: provision one verifier from
whichever registry is reachable, then run the installer with
`LLMLINT_RELEASE_BASE_URL` pointing at your mirror — every byte comes from the
mirror or a registry, and the signature still chains to the Sigstore root.
Elsewhere, `brew install cosign`, a distro sigstore package, or the
[`sigstore/cosign-installer`](https://github.com/sigstore/cosign-installer)
GitHub Action in CI all work too.

You also need a coding harness installed and authenticated (e.g. Claude Code).
See `oneharness list` / `oneharness detect --all`.

## Quick start

```console
llmlint init                 # write a starter llmlint.yml (config-lint plugin on)
llmlint init --with-template # ...and embed the prompt template to customize
$EDITOR llmlint.yml          # write your rules
llmlint                      # lint the configured files
llmlint src/api/**/*.rs      # ...or lint specific files
llmlint --format json        # machine-readable output
```

## Configuration

`llmlint.yml` (discovered by walking up from the working directory; override with
`-c/--config`, repeatable). Discovery is **nested** in both directions. Walking
**up**, *every* config found (one per directory) is merged, nearest first, so a
config beside the files being linted, a project config above it, and a user-level
config higher still layer together — the most-local config wins each top-level
scalar, every config contributes its rules, and a more distant config fills only
the gaps (the same nearest-wins precedence as [plugins](#plugins-shared-rule-sets)).
Walking **down**, a config in a subdirectory governs *its own* part of the project:
its `files` globs are rooted at that directory (a `frontend/llmlint.yml` with
`*.txt` matches `frontend/`'s files, never a same-named file elsewhere), so you can
keep per-area rules next to the code they check. A subtree config scopes *rules*,
not session-wide settings (model, timeout, template, rationales come from the
working directory and up); `--config` replaces the whole walk with no cascade.
`llmlint init` writes it with a leading
`# yaml-language-server: $schema=…` modeline pointing at llmlint's
[published JSON Schema](assets/llmlint.schema.json), so editors with the YAML
language server (e.g. VS Code's [YAML extension](https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml))
give completion and validation as you write. Add the same line to a hand-written
config to opt in.

```yaml
version: 1                     # this config's published version (used when it is consumed as a plugin)

# Files linted when none are passed on the CLI. Omit the whole block (or leave
# `include` empty) to lint every file in the tree from the current directory;
# `exclude` and `.gitignore` still apply.
files:
  include: ["src/**/*.rs"]
  exclude: ["**/generated/**"]

# Require a short `rationale` for every verdict (default true). See Rationales below.
rationales: true

# Default base for `--diff` (when `--diff-base` isn't passed). Any git revision —
# a branch, tag, commit, or `A..B`/`A...B` range. Set it to your default branch so
# a CI quality gate reviews whatever the current branch changed; unset = `HEAD`.
diff_base: main

# Pull in shared rule sets / plugins with one line each. An entry is a local
# path or a URL (`http(s)://`, `file://`); pin a URL to a version with `@`.
plugins:
  - "https://raw.githubusercontent.com/nickderobertis/llmlint/main/assets/config_lint.yml@1"  # bundled: lints this config's own rules
  - "https://example.com/org-rules.yml@1.2.3"   # pinned; fetched + cached once
  - "./team-rules.yml"

# Agents group rules and add reviewer context + harness/model/batch config.
# YAML anchors let you share prompt text with zero framework support.
agents:
  architecture:
    harness: claude-code       # any id from `oneharness list`; omit to use oneharness's own default
    model: opus
    batch_size: 15             # rules per judge run (default 20)
    prompt_template: |         # appended to the master template before render
      You are a senior software architect reviewing service boundaries.

rules:
  - name: handlers_delegate_to_services   # unique, terse, descriptive
    description: |
      true when every HTTP handler delegates business logic to a service layer.
      false when a handler performs business logic (DB queries, domain rules)
      inline.
    agent: architecture        # optional; omit to use the default agent
    # override: true           # optional; extend a same-named plugin rule, inheriting unset fields
    judges: 3                  # optional; independent judges, majority wins (default 1)
    rationale: true            # optional; override the session-wide `rationales` for this rule
    relevance: true            # optional; when to evaluate — see Relevance below (default true)
    files:                     # optional; override the target files for this rule
      include: ["src/api/**"]
```

### Nested & per-directory configs

Configs **nest** — discovery walks both up from the working directory and down
into its subtree, merging every `llmlint.yml` it finds. This lets you layer a
user-level config, a project config, and per-area configs that live next to the
code they govern, with no extra wiring.

```
~/.llmlint.yml                 # user-level defaults (model, rationales…)
my-project/
├── llmlint.yml                # project rules + settings  (run from here)
├── backend/
│   └── llmlint.yml            # rules for backend/**, globs rooted at backend/
│       #   files: { include: ["**/*.py"] }
│       #   rules: [{ name: no_print_debugging, … }]
└── frontend/
    └── llmlint.yml            # rules for frontend/**, globs rooted at frontend/
        #   files: { include: ["**/*.ts"] }
        #   rules: [{ name: no_inline_styles, … }]
```

Running `llmlint` from `my-project/` evaluates **all** of these together:

- `no_print_debugging` runs only on `backend/**/*.py`, and `no_inline_styles`
  only on `frontend/**/*.ts` — each subtree config's `files` globs are **rooted
  at its own directory**, so `**/*.py` under `backend/llmlint.yml` means
  `backend/**/*.py`, never a stray `.py` elsewhere.
- The project's own rules and settings apply across the whole run; a more-local
  config **wins** each setting and can `override` a rule from a config above it.
- **Session settings** (model, timeout, prompt template, rationales) come from
  the working directory and up — a subtree config scopes *rules*, it doesn't
  retune the whole run. Run from `my-project/backend/` instead and that config
  becomes the most-local one, layering under the project and user configs.

Passing **explicit files** (`llmlint backend/svc.py`) narrows this to just the
configs that govern those files: a subtree config is consulted only when a passed
file lives under it, and each rule judges only the passed files inside its own
directory. So `llmlint backend/svc.py` never loads `frontend/llmlint.yml` (nor
fetches its plugins, nor collides with a rule it happens to share a name with) —
you get exactly the rules that apply to what you asked to lint.

Use `llmlint config` to see the merged result and `llmlint config --sources`
(or `llmlint where rules.<name>`) to trace any rule, agent, or setting back to
the exact file it came from. To bypass discovery entirely, pass explicit
configs with `-c/--config` (repeatable) — that roots every glob at the working
directory with no cascade.

### Writing good rules

- **Phrase each rule as a positive invariant.** `holds = true` means the code
  complies; `holds = false` is a violation that llmlint reports and fails on.
- **Make the verdict unambiguous.** Often a plain statement of the property is
  enough — `every public item has a doc comment` already says what passes and,
  by negation, what fails; there's no need to bolt on a "false when…" clause that
  only restates the inverse. Spell out the violating case *only* when it carries
  meaningful detail — concrete examples, easily-confused edge cases — that the
  positive statement leaves unclear. When you do state both, keep them mutually
  exclusive. The bundled config-lint plugin (the `config_lint.yml` URL above)
  lints your config for exactly this, plus descriptive (non-placeholder) names
  that match what each rule checks.
- **Names** are unique, terse, and descriptive (`^[A-Za-z][A-Za-z0-9_]*$`); they
  become the JSON keys of the structured output.
- **Scope a rule by file type or location with `files`, not `relevance`.** When a
  rule applies only to files you can name by path — a file type (`*.py`) or a
  location (`src/**`) — put that subset in the rule's `files` globs. A path scope
  is deterministic and costs no judge tokens, and it keeps the verdict about the
  property, not about applicability. Reserve `relevance` for conditions on a
  file's content or changes that a glob can't express.
- **Scope a rule to the changes it applies to with `relevance`** (see below)
  instead of bolting "…or not applicable" onto the description — that keeps the
  true/false outcome clean and lets llmlint tell "didn't apply" apart from "true".
- **Keep each `description` and `relevance` concise.** A judge call batches an
  agent's rules into one prompt, so tokens one bloated rule spends dilute every
  other rule in that batch. State the invariant in the fewest words that keep it
  unambiguous; add length only when it buys clarity (concrete examples, tricky
  edge cases). config-lint checks this too.

### The prompt template

llmlint renders the judge's system prompt from a
[minijinja](https://docs.rs/minijinja) (Jinja2-style) template. The bundled
default lives in [`assets/default_template.md`](assets/default_template.md); embed
a copy to customize with `llmlint init --with-template`, or set `prompt_template`
yourself. The top-level `prompt_template` *replaces* the master template; an
agent's `prompt_template` is **appended** to it before rendering, so reviewer
context you add per-agent sees the same variables.

These variables are in scope when a template renders:

| Variable | Type | Description |
| --- | --- | --- |
| `files` | list of strings | The target file paths for this run — relative to the working directory, always forward-slashed (so a Windows run reads the same as Linux/macOS). |
| `rules` | list of objects | The rules in this batch. Each has `.name` (the identifier, also the JSON key in the structured output), `.description` (the invariant to judge), `.rationale` (whether this rule wants a justification), `.relevance` (the relevance condition string, or unset for an always-evaluated rule), `.require_line_attribution` (whether every violation must cite a `file` + `line`), and `.files` (the subset of `files` this rule applies to). |
| `file_rules` | list of objects | Per-file applicability — one entry per target file, in the same order as `files`. Each has `.file` (the path), `.mode` (`"include"` or `"exclude"`), `.rules` (the rule names to apply or, when `mode == "exclude"`, to skip — whichever list is shorter), and `.diff` (that file's unified diff, present only under `--diff` when the file changed). The default template's "Target files" section is built from this. |
| `diffs` | list of objects | Per-file changed-line diffs — one entry per *changed* file, present only under `--diff` (empty otherwise). Each has `.file` (matching its entry in `files`) and `.diff` (the unified diff text). The default template inlines these per file via `file_rules`; kept separately for custom templates. |
| `rationales` | bool | True when any rule in this batch wants a rationale — gate the rationale guidance on it. |
| `relevance` | bool | True when any rule in this batch carries a relevance condition — gate the relevance guidance on it. |
| `line_attribution` | bool | True when any rule in this batch requires line attribution — gate the line-attribution guidance on it. |

```jinja
## Target files
{% for f in files %}- {{ f }}
{% endfor %}
## Rules to evaluate
{% for r in rules %}### {{ r.name }}

{{ r.description }}
{% endfor %}
```

A run is one `(agent, file set, judge)` batch, so `rules` is that batch's slice
(see `batch_size`), not necessarily every rule in the config.

### Rationales

By default each judge must justify every verdict with a short **rationale**. The
structured output for each rule is ordered deliberately — the judge echoes the
rule **name**, writes the **rationale**, then commits to the **result**
(`holds` + any `violations`):

```jsonc
{
  "no_inline_sql": {
    "name": "no_inline_sql",                       // 1. anchor on the rule
    "rationale": "raw SQL built inline in db.rs:42, not via the query layer",  // 2. reason
    "holds": false,                                // 3. conclude
    "violations": [{ "file": "src/db.rs", "line": 42, "message": "inline SQL" }]
  }
}
```

Reasoning *before* concluding (and naming the rule first) keeps each verdict
consistent and targeted — it leans on the model's next-token prediction so the
`holds` follows from the evidence just written, not the other way round. Beyond
that, rationales buy you:

- **Auditability** — a durable record of *why* each verdict landed, carried in
  `--format json` for every rule (pass or fail).
- **Debugging** — when a verdict looks wrong, you see the judge's reasoning, not
  just a bare pass/fail.
- **Reliability** — verdicts are measurably steadier when the judge must commit
  to evidence first.

The cost is **extra output tokens on every request**. Turn rationales off to
save tokens:

```yaml
rationales: false            # session-wide default (CLI --no-rationales overrides it)

rules:
  - name: handlers_delegate_to_services
    description: ...
    rationale: true           # …but keep them for this high-stakes rule
```

Precedence, lowest to highest: the session default `rationales` (default `true`)
→ a per-rule `rationale` → the `--rationales` / `--no-rationales` CLI flags
(which set the session default for the run; a per-rule `rationale` still wins).
In the human report, a rule's rationale is shown for every **failure** by
default, and for **every evaluated rule** at `-v`. The default prompt template
asks for rationales that are terse and pithy — the fewest tokens that still cite
the evidence — so the token cost stays small. See
[Cost vs performance](#cost-vs-performance-token-usage) to trade it for a cheaper run.

For a **multi-judge** rule (`judges: N`), the report and `--format json` show
**each judge's** result and rationale, not just one representative — so you can
see exactly where the judges agreed or split:

![llmlint's multi-judge report: a FAIL headed "1/3 judges held" with each judge's held/violated line and rationale, then the pinned violation](docs/screenshots/multi-judge.svg)

### Relevance

Not every rule applies to every change. Rather than make each `description`
carry its own "…or not applicable" escape hatch — which muddies the true/false
outcome and hides *why* a rule passed — declare when a rule should be evaluated
with **`relevance`**:

```yaml
rules:
  # Always evaluated (the default). The judge may not opt out.
  - name: public_items_are_documented
    description: ...
    # relevance: true            # implicit

  # Never evaluated — disabled deterministically, with no judge call.
  - name: legacy_only_check
    description: ...
    relevance: false

  # Conditionally evaluated. The judge decides whether the condition holds for
  # the change *before* the verdict; if it doesn't, the rule is "not relevant".
  - name: errors_are_contextualized
    description: |
      TRUE when every returned error adds context about the operation that
      failed. FALSE when an error is propagated with no added context.
    relevance: the change adds or modifies error handling
```

For a conditional rule the structured output gains a `relevant` boolean, decided
before the verdict — so a not-applicable rule is distinguishable from a true one:

```jsonc
// Not relevant: the object ends after `relevant`; the rationale explains why.
{ "errors_are_contextualized": {
    "name": "errors_are_contextualized",
    "rationale": "the change only renames a struct field; no error handling touched",
    "relevant": false } }

// Relevant: proceed to the verdict as usual.
{ "errors_are_contextualized": {
    "name": "errors_are_contextualized",
    "rationale": "every `?` propagation wraps with `.context(...)`",
    "relevant": true,
    "holds": true } }
```

A **not-relevant** rule is neither a pass nor a violation — it never fails the
build. The human report counts it in a `… not relevant` summary segment and, at
`-v`, itemizes it as a dim `N/A <rule> (not relevant)` line with the reason;
`--format json` carries `"outcome": "not_relevant"` and a `not_relevant` summary
count. For a multi-judge rule, relevance is decided by majority first, then the
verdict is tallied over the judges that found it relevant.

### Ignore directives

Suppress a rule at a specific place with an inline comment in the target file —
the same idea as `# noqa` / `// eslint-disable`, but **strict**: a directive must
name the specific rule(s) and give a reason.

```rust
let q = format!("SELECT * FROM users WHERE id = {id}"); // llmlint: ignore[no_inline_sql] one-off migration, not user-facing
```

```python
# llmlint: ignore-file[public_items_are_documented] generated stubs, documented upstream
```

```rust
// llmlint: ignore-block[no_inline_sql] legacy query layer, migration tracked in JIRA-42
fn legacy_queries() { /* … */ }
// llmlint: ignore-end[no_inline_sql]
```

- `llmlint: ignore[rule, ...] <reason>` is **line-scoped** — it covers the line it
  sits on (a trailing comment) or the line right below it (a comment on its own line).
- `llmlint: ignore-file[rule, ...] <reason>` is **file-scoped** — it covers the
  whole file.
- `llmlint: ignore-block[rule, ...] <reason>` … `llmlint: ignore-end[rule, ...]` is
  **block-scoped** — it covers every line between the open and its matching close.
  The closing `ignore-end` names the same rule(s) and needs no reason. Blocks track
  each rule independently, so rules opened together may be closed at different points
  and blocks for different rules may overlap.

Use whatever comment syntax the file's language uses (`//`, `#`, `/* … */`, `<!-- … -->`);
llmlint keys off the reserved `llmlint: ignore` / `llmlint: ignore-file` /
`llmlint: ignore-block` / `llmlint: ignore-end` prefix.

**Two layers, by design.** llmlint deterministically validates each directive's
*structure* before any judge runs — it must name **specific, configured** rule(s)
and carry a **reason** (except `ignore-end`, which only closes a block). A directive
with no brackets, an empty list, an unknown or misspelled rule, or no reason is a
hard `file:line:` error (exit 2), so a typo fails loudly instead of silently
suppressing nothing. Block pairing is checked too: an unclosed `ignore-block`, an
`ignore-end` with no open block, or re-opening a rule already open is a hard error. Actually *honoring* a
well-formed directive is the judge's job: the default prompt tells it to skip a
named rule's violation at the directive's location. (A custom `prompt_template`
should carry the same guidance if you want directives honored.) Because the
prefix is reserved, a *linted* file that merely documents the feature must use
real rule names or avoid the literal `llmlint: ignore[…]` form.

This structural check is **deterministic and free** — no model call — so it is
also exposed as its own command, [`llmlint check-ignores`](#commands--exit-codes).
Run it in your tight, fast linter loop (next to `cargo fmt` / `clippy`, in a
pre-commit hook, or as a quick CI step), where it catches a typo'd or
reason-less directive in milliseconds. The full `llmlint` run performs the same
check as a pre-flight, so the two never disagree — `check-ignores` just gives you
the fast feedback without waiting on (or paying for) a judge.

### Batching

Model calls are the slow, paid part, so llmlint packs rules into as few as it can.
Rules group **by agent**, then split into batches of at most `batch_size`
(default 20) — one `oneharness run` per batch, over the union of its files, each
rule scoped to its own files in the prompt. Multi-judge rules fan out per judge
(judge `j` runs the rules with `judges >= j`). Fewer, fuller batches, fewer
round-trips.

### Judges and voting

`judges: N` runs a rule through `N` independent judges and takes the **majority**
verdict. `N` must be **odd** (1, 3, 5, …) so the vote can't tie — an even count is
a config error. Only rules that opt in pay the extra cost: judge 1 runs all rules,
judge 2 only the rules with `judges >= 2`, and so on.

### Cost vs performance (token usage)

Defaults favor **judgment quality** over cost: rationales on, a thorough prompt,
every file read in full. Trade some back for fewer tokens, roughly by impact:

- **`judges`** — each extra judge is a full extra pass ([Judges](#judges-and-voting)).
  Keep it at 1 except for high-stakes rules.
- **`rationales: false`** — drops the per-verdict justification, output tokens on
  *every* rule ([Rationales](#rationales)). Re-enable `rationale: true` per rule;
  `--no-rationales` for one run.
- **Fewer agents, bigger `batch_size`** — every batch re-sends the prompt and
  re-reads its files ([Batching](#batching)). Merge rules onto one agent; split
  only for a different harness, model, or reviewer context.
- **Read less** — narrow `files.include`/`exclude`; `--diff` reviews only changed
  lines; `FILES`/`--rule`/`--agent` lint a subset.
- **`require_line_attribution`** off unless you need pinned locations — on can
  trigger localize re-prompts.
- **`oneharness.schema_max_retries`** — caps re-asks on a schema-invalid answer.
- **`model`** — dollars, not tokens: a cheaper model per agent or run.

`llmlint check-ignores` spends no tokens at all.

### Results logging & history

The terminal report is deliberately terse — failing rules and a summary. But the
full results (every rule's verdict, votes, per-judge breakdown, violations, and
rationales) are worth keeping. **Results logging is on by default:** each
`lint`/`lint-config` run is also written to disk as one JSON record under an
auto-generated, time-sortable id, and the run prints that id on stderr:

```console
$ llmlint
FAIL no_inline_sql
     src/db.rs:12: inline SQL

2 rules: 1 passed, 1 failed, 0 skipped
See full results with `llmlint history 20260704T153000Z-1a2b3`
```

Inspect what was logged with `llmlint history`:

```console
llmlint history                       # list recent runs (most recent first)
llmlint history latest                # full results of the most recent run
llmlint history <id>                  # full results of a specific run
llmlint history <id> --status fail    # only the failing rules
llmlint history <id> --rule my_rule   # only one rule
llmlint history <id> --path           # just the JSON record's path (for scripting)
llmlint history <id> --format json    # the raw record
llmlint history --format json         # a JSON array of run summaries
```

Records live in the platform per-user **data** directory by default
(`~/.local/share/llmlint/history` on Linux, `~/Library/Application Support` /
`%LOCALAPPDATA%\llmlint\data\history` on macOS/Windows; `$XDG_DATA_HOME` is
honored). Only the newest **100** runs are kept; older records are pruned after
each run. All three knobs are configurable:

```yaml
history:
  enabled: true          # false turns logging off entirely
  max_runs: 100          # how many recent runs to keep
  dir: .llmlint/history  # where records go (relative to nothing in particular; an absolute path is clearer)
```

The `LLMLINT_HISTORY_DIR` environment variable overrides `dir`, and `--no-history`
(or `LLMLINT_NO_HISTORY=1`) disables logging for a single run. Like the other
top-level settings, `history` is a cwd-and-up **session setting** (a subtree
config never retunes it) and traces through `llmlint config --sources` / `llmlint
where history.dir`.

### oneharness passthrough

llmlint lets oneharness discover its own `oneharness.toml` by default. To force a
specific oneharness config, use `--oneharness-config <path>` (or `oneharness.config`
in the llmlint config); it is forwarded via oneharness's `--config`. Override the
binary with `--oneharness-bin` or `$LLMLINT_ONEHARNESS_BIN`.

### Plugins (shared rule sets)

`plugins` pulls other llmlint configs into this one — their rules and agents are
merged in. For the **top-level settings** (template, files, oneharness,
rationales), **the nearer config to the root wins**: your config's settings take
precedence over a plugin's, a plugin's over its own plugins', and an
earlier-listed plugin over a later sibling. A plugin only *fills in* a setting
the including config left unset, so a shared plugin can ship sensible defaults
without overriding what you set locally. The CLI overrides all of them (see
Commands). Each entry is a config file:

- a **local path** (`./team-rules.yml`), resolved relative to the including file;
- a **URL** — `http(s)://` (fetched over HTTPS) or `file://` (read directly).

Resolution is **transitive**: a pulled-in config's own `plugins` are pulled in
turn, and so on. Diamonds and cycles are de-duplicated (each config loads once),
and the chain is bounded at a depth of 100 to fail fast on a pathological graph.

By default a rule name is **unique** across the whole merged config — declaring
the same name twice is an error. To **adjust** a rule a plugin gave you without
restating it, re-declare it with `override: true` and set only the fields you
want to change; every other field (including the `description`) is inherited from
the plugin's rule:

```yaml
plugins:
  - "https://example.com/org-rules.yml@1"   # ships `no_inline_sql`, 1 judge

rules:
  # Keep the org rule's text, but vote it across 3 judges and scope it tighter.
  - name: no_inline_sql
    override: true
    judges: 3
    files:
      include: ["src/db/**"]
```

The override must be set on the **nearer-root** config, and there must be exactly
one base rule (the same name declared *without* `override`) for it to extend — an
`override` with nothing to override is an error, so a typo'd name can't silently
do nothing. When several configs override the same base, the nearest-root
override wins each field.

URL fetching is built in (a pure-Rust HTTPS client — no `curl` or other external
tools, no system OpenSSL) and honors the standard `HTTP(S)_PROXY` / `NO_PROXY`
env vars. The bundled config-lint plugin ships inside the binary and resolves
**offline**.

A URL may be **pinned to a version** with an `@` suffix matching the plugin
config's own top-level `version`: `@1` accepts any `1.x`, `@1.2` any `1.2.x`,
`@1.2.3` exactly that. The pin is both an assertion (a mismatch is a hard error)
and the **cache key**: a pinned URL is fetched once into the cache and reused on
later runs without refetching — bump the pin to pull a new version. An *unpinned*
URL is fetched every run.

The cache lives under `$XDG_CACHE_HOME/llmlint/plugins` (override with
`LLMLINT_CACHE_DIR`). Set `LLMLINT_PLUGIN_REFRESH=1` to force a refetch.

### Linting your llmlint configs

llmlint ships with a **config-lint** rule set that lints llmlint config files
themselves — that every rule's `description` yields a clear, unambiguous verdict,
its `name` is descriptive (non-placeholder) and matches what the description
checks, a conditional rule uses `relevance` instead of bolting "…or not
applicable" onto the description, and a rule scoped to a file type or location
uses `files` globs rather than `relevance`. It's the [Writing good rules](#writing-good-rules)
guidance, enforced (and each rule is phrased to pass its own checks). Each finding
cites the config file + the offending rule's line (`require_line_attribution`).
There are two ways to use it:

- **As a plugin** — add the bundled URL to your `plugins` (it's on by default in
  `llmlint init`), and its rules run against your config files on every normal
  `llmlint` run, alongside your own rules:

  ```yaml
  plugins:
    - "https://raw.githubusercontent.com/nickderobertis/llmlint/main/assets/config_lint.yml@1"
  ```

  The URL ships **inside the binary** and resolves offline (no network, no
  cache), so it works disconnected and needs no pin bump to stay current.

- **As a subcommand** — `llmlint lint-config` is the `lint` command with that
  plugin included by default, so you don't have to add it to your config. It
  first runs the deterministic ignore-directive (comment) check over the config
  files, then judges each config's rules. Point it at specific files
  (`llmlint lint-config path/to/llmlint.yml`) or let it discover every llmlint
  config in the tree. Handy in CI as a standalone "is my config well-authored?"
  gate.

### Finding where something is defined

Once configs merge across files and plugins, a rule, agent, or setting in the
effective config can come from any of them. Two commands trace an item back to
the file (or plugin URL) you'd edit to change it.

`llmlint where <path>` answers one lookup and prints **just the source**, so it
composes in scripts. The path mirrors the config structure:

```console
$ llmlint where oneharness.model            # a top-level setting
./shared/team.yml
$ llmlint where agents.security             # an agent
./shared/team.yml
$ llmlint where rules.no_inline_sql         # where a rule is defined
https://example.com/org-rules.yml@1
$ llmlint where rules.no_inline_sql.judges  # the file an override set a field in
./llmlint.yml
$ editor "$(llmlint where rules.no_inline_sql.judges)"
```

Because an `override` resolves **field by field**, a single rule can draw its
`description` from the plugin that defined it and its `judges` from your config —
`where rules.<name>.<field>` points at the file that actually set that field (or
the definition site when no override did). An unknown name lists what's available,
and a setting left at its built-in default says so, both exiting non-zero.

For the whole picture at once, `llmlint config --sources` adds a `sources` block:

```jsonc
{
  "config_files": ["./llmlint.yml", "./shared/team.yml", "https://example.com/org-rules.yml@1"],
  "sources": {
    "settings": { "version": "./llmlint.yml", "oneharness.model": "./shared/team.yml" },
    "agents":   { "security": "./shared/team.yml" },
    "rules": {
      "no_inline_sql": {
        "source": "https://example.com/org-rules.yml@1",  // where the rule is defined
        "fields": { "judges": "./llmlint.yml" }            // a field an override moved
      }
    }
  },
  "config": { /* … the merged config … */ }
}
```

A rule with no cross-file override has no `fields` entry; settings and agents are
each kept whole from the nearest-root config that set them, so they have a single
source.

## Commands & exit codes

- `llmlint [FILES...]` — lint (the default). `--format human|json`, `--agent`,
  `--rule`, `--max-parallel`, `--timeout`, `--cwd`. Target individual rules with
  `--rule NAME` (repeatable) or a whole group with `--agent NAME`; an unknown
  rule/agent name is an exit-2 error that lists the available names. Every
  top-level setting also has a flag that wins over the config:
  `--rationales`/`--no-rationales`, `--model NAME`, `--schema-max-retries N`,
  `--prompt-template PATH`, plus `--oneharness-bin`/`--oneharness-config`. Pass
  `--diff [<backend>]` to add each changed file's diff to the judge prompt so it
  reviews only the changed lines; bare `--diff` uses the `git` backend (compared
  against `HEAD`). Add `--diff-base <REF>` to compare against a different git
  revision instead of `HEAD` — a branch, tag, commit, or `A..B`/`A...B` range —
  so `--diff --diff-base main` reviews exactly what the current branch changed
  versus `main` (the PR-review case). The base can also be set once in config as
  `diff_base:` (the flag overrides it).
- `llmlint check-ignores [FILES...]` — validate the *structure* of inline
  `llmlint: ignore` directives in the target files, **deterministically and with
  no model call** (`-c/--config`, `--cwd`; pass `FILES` to scope it, e.g. the
  changed files in a pre-commit hook). This is the same pre-flight `lint` runs,
  split out for the fast static-check loop: exit `0` when every directive is
  well-formed, exit `2` (located `file:line:`) on a typo'd / reason-less /
  unbalanced one.
- `llmlint lint-config [FILES...]` — lint llmlint config files with the bundled
  [config-lint](#linting-your-llmlint-configs) rules, without adding the plugin to
  your own config. It's the `lint` engine with that plugin forced on: it first
  runs the deterministic comment (ignore-directive) check, then judges each
  config's rules. Shares `lint`'s flags (`--format`, `--model`, `--timeout`,
  `--cwd`, `--diff`, …); the config source is fixed, so `--config`/`--agent`/
  `--rule` aren't taken. Same exit codes as `lint`.
- `llmlint init` — write a starter config (`--with-template`, `--global`, `--force`).

  ![llmlint init writing a starter llmlint.yml](docs/screenshots/init.svg)
- `llmlint config` — print the merged config and the ordered list of sources that
  contributed, as JSON. Add `--sources` to also trace every rule, agent, and
  setting back to the file (or plugin URL) it came from — see
  [Finding where something is defined](#finding-where-something-is-defined).

  ![llmlint config printing the merged config and its sources as JSON](docs/screenshots/config.svg)
- `llmlint where <path>` — print the single source of one config item: a setting
  (`oneharness.model`, `version`), `agents.<name>`, `rules.<name>`, or a rule
  field `rules.<name>.<field>`. See
  [Finding where something is defined](#finding-where-something-is-defined).
- `llmlint doctor` — check that oneharness is installed and reachable.

  ![llmlint doctor reporting the resolved oneharness version](docs/screenshots/doctor.svg)
- `llmlint history [ID]` — inspect [logged run results](#results-logging--history).
  With no id, list recent runs; with an id (or `latest`), show that run's full
  results. `--status pass|fail|skipped|not_relevant` and `--rule NAME` (both
  repeatable) drill into part of a run; `--path` prints just the record's file
  path; `--format json` emits the raw record (or a JSON array when listing);
  `--dir`/`--cwd`/`--limit` tune where and how much.

Exit codes: `0` all rules hold · `1` at least one violation · `2` usage,
configuration, or harness error (could not complete the lint).

## Development

```console
just bootstrap   # toolchain components + fetch (from a clean clone)
just check       # full gate: fmt, clippy -D warnings, tests + 95% coverage, docs
just test-e2e    # the e2e binary journeys in isolation
just deps-check  # cargo deny + cargo machete
just lint-live   # opt-in: ad-hoc lint against the REAL oneharness + a real harness
just live-claude # opt-in: live e2e — built llmlint → real oneharness → real harness
```

Tests drive the real `llmlint` binary against a hermetic mock-oneharness fixture.
The live tier (`just live-claude`, and the ad-hoc `just lint-live`) drives the
whole stack end to end against a real, authenticated harness — the only thing that
makes real model calls, and out of the `check` gate. It runs on PRs in its own
workflow across Linux/macOS/Windows, so a missing CLI, auth, or oneharness is a
hard failure, not a skip. See `AGENTS.md` and `tests/AGENTS.md`.

## License

MIT — see [LICENSE](LICENSE).

