Metadata-Version: 2.4
Name: ruleprobe
Version: 0.3.0rc1
Summary: Find out which of your agent rules actually fire: deterministic detectors over your coding agent's own transcripts
License-Expression: MIT
Project-URL: Homepage, https://github.com/JakeSelby/ruleprobe
Project-URL: Documentation, https://ruleprobe.jakeselby.com
Project-URL: Issues, https://github.com/JakeSelby/ruleprobe/issues
Keywords: agents,transcripts,claude-code,codex,measurement,rules
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# ruleprobe

Find out which of your agent rules actually fire.

You have written rules for your coding agent. A `CLAUDE.md`, an `AGENTS.md`, a house style
your team argued over. You do not know whether any of them changed what the agent did.
ruleprobe reads the transcripts your agent already wrote and counts the observable things
that happened in them.

## Sixty seconds

```sh
uvx ruleprobe report
```

That reads `~/.claude/projects/**/*.jsonl` (and `~/.codex/sessions/**/*.jsonl` and
`~/.gemini/tmp/*/chats/`, if Codex or Gemini CLI wrote any), runs the detectors over them, and
prints:

```
detector                                 hits  sessions    of    share  note
----------------------------------------------------------------------------
cache-hygiene/compact                       0         0   339       0%  unobserved
cache-hygiene/model-switch                 11        11   339       3%
secrets/secret-in-write                     3         2   339       1%
transcript-hygiene/unfiltered-find          0         0   339       0%  unobserved
transcript-hygiene/whole-file-cat          89        22   339       6%
verification/no-verify                      0         0   339       0%  unobserved
```

That is a real run, over one week of one person's transcripts. Two detectors have never
fired on this machine, which is itself the finding: `unfiltered-find` and `no-verify` are
guarding against something that is not happening, and the rules behind them are paying rent
in the context window for nothing.

`share` is the fraction of sessions the detector fired in at least once. `unobserved` means
it has never fired in the window. `frequent` means it fired in more than 30 percent of sessions;
it describes the share and advises nothing. Both notes stay blank until there are twenty
measured sessions, because a share over five sessions is noise.

With no `--rules`, `report` also finds your rule files itself and prints a coverage block under
the table, saying which rules a detector measures. It reads `~/.claude/CLAUDE.md`,
`~/.codex/AGENTS.md` (or `AGENTS.override.md`) and `~/.gemini/GEMINI.md`, and at each working
directory a transcript recorded, the project files that runtime reads there: `CLAUDE.md`,
`.claude/CLAUDE.md`, `CLAUDE.local.md` and `.claude/rules/**/*.md` for Claude Code, `AGENTS.md`
(or `AGENTS.override.md`) for Codex, `GEMINI.md` for Gemini CLI. A Claude Code or Gemini file's
`@path` imports of other markdown files are followed, five hops at most. A file reached twice,
or with the same bytes as another, as in several worktrees of one repository, is read once, and
each rule is named by where its file is, `~/work/app/CLAUDE.md#testing`. The rule text is
today's, not what was in force when an older session ran, and the block says so. A found file
may be a clone of somebody else's repository, so it binds through the catalog alone: its front
matter is ignored, and only a regular file of at most 1 MB is read, at most 200 from one rules
folder. Every file must resolve inside its project, a global file anywhere under the home
folder, and an import that leaves its project, or the runtime's global folder for a global
file, is refused; linked folders are not walked. Refused and skipped files are counted in the
block, never named.
Parent directories are not searched. `--rules DIR` reads that directory instead, and `--no-config`
reads no rule file.

**Bind your rules.** The catalog binds a section only when a sentence says exactly what a
detector counts, so most sections of a real rule file start unmeasured. A few minutes with
`ruleprobe bind` closes that gap with bindings you choose:

```sh
uvx ruleprobe bind --plan > plan.yaml     # each unmeasured section, its nearest detectors, why
# set `bind:` to a detector id on the sections you want measured, then:
uvx ruleprobe bind --apply plan.yaml      # records them in ~/.config/ruleprobe/bindings.yaml
```

The plan lists every unmeasured section in sorted order, with the catalog detectors its
sentences came nearest to, ranked by how many sentences matched, and the reason it did not bind:
an exception such as "unless", a condition, or a sentence matching two entries. Nothing stops you
binding a section the catalog refused; the plan tells you what you are overriding. `--json`
prints the same plan as data, and `--apply` takes either form. Pass the same `--rules` or
`--root` to both. `--apply --project` writes the current repository's `.ruleprobe/bindings.yaml`
instead, with paths relative to it; your rule files are never edited.

A bound rule reads `measured  user-bound, <detector>` in the coverage block, apart from
`catalog-bound`, because a person made that claim. Each binding keeps a sha256 of every line of
the section, and when anything but line endings or trailing spaces changes, a blank line, a
quote, table or code block included, the
rule reads "binding stale, run ruleprobe bind" and is not counted as measured until you bind it
again. `report` reads only your global bindings file and the one in the project you run it from,
or that `--rules` is in, looking no higher than the repository root; a bindings file in a project
found through the transcripts, which may be somebody else's clone, is never read, only counted.
`--apply` never writes through a linked folder, and never rewrites a bindings file you added
comments or other keys to; run one apply at a time.
A binding names a shipped, catalog or detector-file id and never defines a detector; an unknown
id is refused. `--no-config` reads no bindings.

Nothing is sent anywhere, no model is asked anything, `report`, `explain` and `bind --plan`
write nothing, `label` writes only under the directory you name, `bind --apply` writes only its
bindings file, and the same transcript gives the same answer every time. Python 3.9 or newer,
standard library only.

Other groupings, and a window:

```sh
uvx ruleprobe report --by repo --since 30      # last 30 days, one line per repository
uvx ruleprobe report --by stance --stance commits=conventional   # grouped by configuration
uvx ruleprobe report --root ./transcripts      # a directory of your own
uvx ruleprobe report --rules ./docs/rules      # bind detectors to rule files, and name the gaps
uvx ruleprobe bind --plan --rules ./docs/rules # the unmeasured sections, to bind yourself
uvx ruleprobe detectors                        # what would run
uvx ruleprobe corpus                           # how good each detector is, over the labelled corpus
uvx ruleprobe report --json                    # the same numbers as data, rows included
```

Each row and the `--json` result carry `schema_version`, and a row with a version this release
does not know is left out of every count and tallied in `unknown_schema`. The result also
carries `renamed`, the effective fold map of retired detector id to current id that its counts
were read through. It is complete, so a saved result folds a retired id by applying it as it
stands, never by merging it again with a later release's shipped map.
The result also carries `coverage`, on every run: `measured`, `dark` and `unmeasured` count
rules, not sessions as the top-level `measured` and `unmeasured` do, and `share` is the
measured share of them; `catalog` counts the measured rules the shipped catalog binds, and `user`
the ones a binding you confirmed binds. With no rules read, the counts are zero and `share` is
`null`.

A row names each detector that raised in `rules_errors`, and that session leaves the
detector's hit denominator only. A row also carries `compliance`, detector id to
`{"opportunities", "followed", "undecided"}`, for each enabled detector that defines
`opportunities`; an undecided opportunity counts in `undecided` alone. A point that names a
`tool_use_id` appears at most once per turn, and a repeat is malformed; an id-less point,
`None` or `""`, counts once per occurrence. When that callable raises, or returns something
other than `(turn, tool_use_id, followed)` triples (recorded as `MalformedOpportunities`),
its `rules_errors` entry carries `"hook": "opportunities"`: the detector loses its
`compliance` entry, and its hit figures are untouched.

The report sums `compliance` beside hits per session, never in place of them. A detector that
has any adds `opportunities`, `followed`, `undecided` and `rate` columns to its line, and
`opportunities`, `followed`, `undecided` and `compliance_rate` to its `--json` entry; one
without has none of them. `compliance_rate` is `followed / opportunities`, and it stays `null`,
printed `-`, until the line holds twenty opportunities (`--min-opportunities N`); undecided
points never count toward that. `--by repo` and `--by stance` group compliance as they group
hits, under each group's `compliance` key and its own minimum. Retired ids fold as hits do. A
session whose `opportunities` failed, or whose detector raised, is out of that detector's
compliance figures, as is one whose stored `compliance` entry cannot be read;
`opportunity_errors` and `malformed_compliance` count those sessions per detector, and a note
names them.

A transcript does not record the configuration it ran under, so `--stance dimension=variant`
is how you say what it was. It is repeatable, it is what `--by stance` groups on, and it is
what a detector's `gate:` block reads: a gated detector with no stance passed never fires,
and `ruleprobe detectors` names the stance each one is waiting for.

## Sixty seconds on a rule of your own

The six above are generic. Your rules are not, but a rule in a common shape needs no detector
at all. Point `--rules` at your rule files and a section that no front matter binds is matched
against a small shipped catalog by what it says, with no model: its heading and each sentence
of its text are tested against one anchored pattern per entry, a sentence that matches
exactly one entry binds that entry's detector, and a section is measured by every detector its
sentences bind. In a clone:

```sh
ruleprobe report --root docs --rules tests/fixtures/catalog
```

binds a file holding one section in each shape, and lists each section as `measured` with
the note `catalog-bound, <detector id>`. The shapes are: run the tests before finishing
(`testing/test-after-change`), never skip git hooks (`verification/no-verify`), never
force-push the default branch (`git-safety/force-push-default`), use uv, not pip
(`package-manager/pip-install`), do not read a whole file
(`transcript-hygiene/whole-file-cat`), Conventional Commit subjects
(`commits/non-conventional-subject`), never commit a secret-shaped file
(`secrets/secret-file-add`), never compact the context (`cache-hygiene/compact`),
never switch models mid-session (`cache-hygiene/model-switch`), never write a secret into a
file (`secrets/secret-in-write`), and filter every `find` (`transcript-hygiene/unfiltered-find`).
The first seven read the common ways of saying them, not one wording each: "Before you hand
back, run the tests", "Run the full test suite after every change", "Git hooks must always run;
never skip them", "Main must never be force-pushed", "Prefer uv over pip", "Do not use pip; use
uv", "Read narrowly: reach for head, grep or a sed range rather than cat", "Write commit
messages in the Conventional Commits style" and "Keep credentials out of source control" all
bind. Each binds only a sentence it covers whole, apart from a closing "no exceptions", so
anything more - a scope, a destination, an aside in parentheses or after a dash, a second
clause - leaves the rule unmeasured: "Never force-push on Fridays to main", "Use uv in CI, not
pip", "Use uv, not pip, in production" and "Never commit .env files (in the public repo)" bind
nothing. So do rules narrower than their detector by name: "Never skip pre-commit hooks" leaves
out the pushes `verification/no-verify` counts, and "Never commit .env files" or "Never check in
private keys" names one kind of the secret-shaped files `secrets/secret-file-add` counts.
The last four bind the shipped detectors of the same ids, and only a rule as wide as what the
detector counts. Their patterns must reach the end of the sentence, and name a session rather
than a task, files in general rather than a commit, the repository, a config file or the code,
since the detector counts a secret written to any file, a gitignored `.env` included, and the
context's compaction rather than the typed `/compact` or compaction alone, since the detector
counts automatic compactions and a database compacts too. So "never compact more than once per
session", "never switch models mid-task", "never run a bare find from the home directory" and
"never put secrets in a commit" bind nothing rather than a detector that would count what the
rule allows, and "do not change the model" alone is read as a data model and binds nothing.

One row reads the other way round. Every other row counts violations, so fewer hits is
better; `testing/test-after-change` counts compliance - a hit is a file change that a later
test run followed - so more is better, and its opportunities and followed count are the
measure. Its errors run low: a test run it does not recognise leaves the change counted as not
followed, so read its followed share as a floor. Its description lists the runners it misses.

The binding under-counts on purpose. A sentence matching no entry, or more than one, binds
nothing, and a section none of whose sentences binds stays unmeasured; the coverage block names
the entries when a sentence matched several, and when a sentence matched an entry and a word
below unbound it, it names that word and the entry, its nearest, as in
`a catalog shape with an exception or condition (unless), nearest git-safety/force-push-default`.
A section whose sentences bind different entries
is measured by each, and the coverage block lists them all: "Never force-push to main. Use uv,
not pip." binds both. A pattern never reaches across a clause break (`;`, `,`, `:`, a run of
hyphens between spaces such as ` - ` or ` -- `, a dash), except inside a closed form of its own
rule: the comma of its contrast ("use uv, not pip"), the comma after a fronted clause ("before
you hand back, run the tests"), the semicolon before its restatement ("do not use pip; use uv")
and the colon of a closed lead-in or a commit format. An exception or a permission - except (excepted, exception), exempt
(exempted, exempting, exempts, exemption), unless, other than, apart from, excluding, allowed,
fine, okay, ok - anywhere in a section, its heading included, unbinds every rule in it, so "Never
force-push to main. Hotfixes excepted." is left unmeasured rather than read as a rule it is not,
and so is a list under "Except on release branches:". A narrower reach was tried; every markdown
shape - a lead-in, a list item of two paragraphs, an abbreviation - carried an exception past it.
A sentence does not end inside parentheses or after e.g., i.e., etc., vs. or cf. A negated marker - no exception, without exception, without any exception, and admit,
allow, make or with no exception, singular or plural - grants nothing where its clause ends, so
"Never force-push to main. No exceptions." binds, while "without exception approval" does not
count as one. A condition or a contrast - if, when, but, however, without - unbinds only the
sentence it is in: "Never force-push to main when others share it" binds nothing, while "Run
the tests before finishing. If one fails, fix it." binds; a sentence split at an abbreviation
not listed above can carry a condition out of the rule's sentence, a known over-count. `only`
is not a marker: it intensifies as often as it narrows. A catalog detector joins the report only when a rule binds it, a detector of your
own with a catalog id replaces it, `ruleprobe corpus` scores every entry, and
`ruleprobe detectors` lists every entry marked `catalog`.

One place it over-counts, knowingly. A file with no heading, or none of whose sections is a
rule, is one rule and binds by its whole text: one matching sentence marks the whole file
measured, and its other sentences may say "if" or "when" as they like, so the measured share
counts it as one measured rule while the detector measures one sentence of it. The exception
and permission words still apply to the whole text, so such a file that says "fine" or "unless"
anywhere stays unmeasured - in a long file, the likelier outcome.

A catalog entry measures the shape it names and no more. "Never commit secrets" binds
`secrets/secret-file-add`, which counts `git add` of a secret-shaped file; a secret written
inline into a file or a command is not what it counts. "Never write API keys to a file" binds
`secrets/secret-in-write`, which counts a live credential a Write or Edit writes, or a heredoc
carries; a key on a command line outside a heredoc is not what it counts.

Any other rule takes no Python either: write a detector beside it as data. This is a real
run over the example transcript and the example rules in this repository, so it is
reproducible from a clone:

```sh
cat docs/rules/house-style.md
```

```markdown
---
rule: house-style
detector:
  id: house-style/sudo-install
  event: tool_use
  when:
    command: {starts_with: [sudo, pip]}
---

# House style

Install with `uv`, never with `sudo pip`.
```

```sh
ruleprobe report --root docs --rules docs/rules
```

```
detector                                 hits  sessions    of    share  note
----------------------------------------------------------------------------
cache-hygiene/compact                       0         0     1       0%
cache-hygiene/model-switch                  0         0     1       0%
house-style/sudo-install                    2         1     1     100%
secrets/secret-in-write                     0         0     1       0%
transcript-hygiene/unfiltered-find          0         0     1       0%
transcript-hygiene/whole-file-cat           0         0     1       0%
verification/no-test-run                    1         1     1     100%
verification/no-verify                      0         0     1       0%

rules: 2 measured, 1 dark, 1 unmeasured (50% measured)
  measured   house-style                    docs/rules/house-style.md
  dark       secrets                        docs/rules/secrets.md: a credential that never reaches a file leaves no shape in a transcript
  measured   verification                   docs/rules/verification.md
  unmeasured working-style.md#working-style docs/rules/working-style.md
```

Three lines of that report are the point. `house-style/sudo-install` is a rule of the
reader's own, firing. `secrets` is **dark** by choice: its front matter carries
`opt_out: <reason>`, because the rule is about a credential that never reaches a file and a
transcript only shows what did. `working-style` is **unmeasured**: it has neither a detector
nor an opt-out, so it is split at its headings and its one section is listed by its id,
`working-style.md#working-style`; saying so is the only way an author sees the gap. Nothing
fails; a report is evidence, not a gate.

Point `--rules` at whatever directory your own rules live in, and drop the same entries into
`.ruleprobe/detectors.yaml` at the root of a repository, or into
`~/.config/ruleprobe/detectors.yaml` for the ones you want everywhere. Both are found
without a flag; `--no-config` skips them.

## Writing a detector

An entry is `id`, `rule`, `event`, `when`, and an optional `gate`. `event` is one of
`tool_use`, `assistant_text` and `session`, and it says what a hit is counted against. `when`
is a matcher: a mapping in which every key must hold, composed with `any`, `all` and `not`.

An entry may also carry `schema_version`, the integer schema it was written under; this
release reads 1 and 2. An entry without one takes its detectors file's top-level `version`,
and a file without one is schema 1. Only a detectors file has a top-level `version`; a
`detector:` entry in rule-file front matter has no file-level default, so without its own key
it is schema 1. In a detectors file the file-level key is `version`: a top-level
`schema_version` is a finding, as a bad `version` is. An entry's own key wins over the file's.
A value this release does not know - newer than 2, below 1, or not an integer (`"2"`, `2.0`
and `true` included) - is a finding with its file and line, and only the entries that would be
read under it are skipped, so an entry written for a later schema is refused rather than
counted wrong.

```yaml
version: 2
detectors:
  - id: house-style/sudo-install
    when:
      command: {starts_with: [sudo, pip]}
  - id: house-style/old-entry
    schema_version: 1
    when:
      command: {name: npm}
```

**A tool use.** The shape is a command, a tool name, or an argument:

```yaml
- id: house-style/wide-grep
  rule: house-style
  event: tool_use
  when:
    command:
      name: grep
      none_of: [-n, --include, --exclude]
      arg_count: {max: 1}
```

**An assistant message.** One hit per message the pattern matches:

```yaml
- id: voice/hedged-verdict
  rule: voice
  event: assistant_text
  when:
    message:
      role: assistant
      final: true
      regex: "(?i)(should (now )?work|I think it works)"
```

**A whole session.** `absent` counts a session in which something never happened, which is
the only way to measure a rule that asks for something to be done; `order` counts one event
followed by another; `change` counts a field that differs from the event before it:

```yaml
- id: verification/no-test-run
  rule: verification
  event: session
  when:
    absent:
      of:
        command: {name: [pytest, tox, nox]}
      scope: session
```

Over a Bash command the shell parse skipped - empty or missing, longer than 16,384
characters, or one that does not tokenize or parse - the segment keys of `command`, `git` and
`env`, and a `text` read of `source: heredocs`, are undecided rather than false, so `not` and
`absent` do not count it. Any one such command in a session leaves the session-scope `absent`
above undecided, and so no hit, whether or not that command was the test run.
`command: {unparsed: true}` on its own matches those commands; combined with a segment key it
can never fire.

The matchers, in one list: `tool` (`name`, `glob`), `arg` (`field`, `regex`, `path_glob`,
`contains`, `equals`, `exists`), `command` (`name`, `starts_with`, `contains`, `none_of`,
`arg_count`, `sole_segment`, `redirect`, `unparsed`, `regex`, `program`, `first_operand`),
`git` (`subcommand`, `args_any`, `args_none`, `token_prefix`, `arg_regex`, `config_regex`,
`message_regex`),
`env` (`name`, `command`), `text` (`source`, `regex`, `contains`), `message` (`role`,
`final`, `regex`, `contains`), `kind`, and the three session matchers `order`, `absent` and
`change`. `ruleprobe/matchers.py` documents each in one line. The shipped six in
`ruleprobe/detectors/common.yaml` use ten of them - `tool`, `arg`, `command`, `git`, `env`,
`text`, `kind`, `change`, `any` and `all` - because that is what those six observables need;
`message`, `order`, `absent` and `not` are exercised by the examples on this page and in
`tests/`, not by a shipped detector.

Three rules about the format worth knowing before you hit them. Every key inside one
`command` block is read against the *same* pipeline segment, so two constraints on one
command belong in one block rather than in an `all` of two. A session matcher may only be
the whole of a `session` detector's `when`, because a hit it produces is not a hit on an
event in hand. And a list is always alternatives: `regex`, `contains` and `path_glob` hold
when any one of their patterns does. `contains` is a substring, so `contains: no-verify`
finds the token `--no-verify`; `path_glob` is a path, so `*` stops at a `/`, `**` crosses
one, and `src/*.py` matches the absolute path a transcript actually carries.

**The format is a YAML subset, and JSON is the same thing.** YAML is not in the standard
library and this package takes no dependencies, so `ruleprobe/declarative.py` implements the
subset a detector needs - block and flow mappings and sequences, scalars, quoted strings,
comments - and refuses everything else by name and line: anchors, aliases, tags, block
scalars, directives, and more than one document in a file. A file named `.json` is read by
the standard library's JSON parser into exactly the same objects, so a generator can write
JSON and a person can write YAML.

**A malformed entry is a finding, not a crash.** It is printed with its file, its line and
its reason, that entry is skipped, and every other detector in the file still runs.

## What it actually covers

Be clear-eyed about the scope, because the name promises more than version 0.1 delivers.

**Six detectors ship with the package**, and they are the generic ones: reading a whole
file into the context window, an unfiltered `find`, a commit or push that walks past the
repository's hooks, a live credential written to a file, a context compaction, and a
model change mid-session. They are in the package because they mean the same thing in every
repository, and none of them needs to know what your rules say.

**Your own rules take a detector you write**, as data in the declarative format above or,
when the shape is past what a matcher can say, as Python:

```python
from ruleprobe import DEFAULT, Detector, iter_sessions, measure, report

def sudo_install(events, ctx):
    return [(p.turn, p.id) for p in ctx.bash
            if any(seg[:2] == ["sudo", "pip"] for pipe in p.pipelines for seg in pipe)]

DEFAULT.add(Detector("house-style/sudo-install", "house-style", "bash", sudo_install))
print(report([measure(s) for s in iter_sessions(since=30)]))
```

`measure`, `DEFAULT`, `Detector` imported from the root, and `Parsed`'s `.turn`, `.id` and
`.pipelines` are importable but not declared (see [the public API](#the-public-api)). A tool
pinning a version has declared alternatives for some: `Detector` from `ruleprobe.registry`,
`hit(p.event)` for `(p.turn, p.id)`, and `pipelines(p.command)` for `p.pipelines`. No declared
function yet produces a row, so `measure` has none.

The two are the same engine: `ruleprobe/detectors/common.yaml` is the shipped six written as
data, and a test asserts it produces hit-for-hit what the Python in
`ruleprobe/detectors/common.py` produces over the corpus. Python remains the escape hatch,
and the seam a third compiler plugs into is still `register_compiler` and `from_spec`.

**What a matcher cannot say.** It reads one event, or one of the three session shapes, and
nothing else. There is no arithmetic, no counting ("more than three reads in a turn"), no
reading a tool's *result*, no comparing one argument with another, and no state carried
across turns beyond `order`, `absent` and `change`. The `command` matcher inherits every
known miss of the shell parse in `ruleprobe/shell.py` - a command inside a substitution is
invisible, and so is a variable's value. A rule whose shape needs any of that is a Python
detector, and the report will not pretend otherwise.

**Unbound rule files split into one rule per section.** A file whose front matter gives
`detector:` or `opt_out:` a value is one rule, named by its `rule:` key or its file name; a key
with no value binds nothing and is a finding. Any other file is split at its ATX (`#`)
headings, so a `CLAUDE.md` of twelve sections is twelve entries in the coverage block, each
with the id `<path>#<heading-slug>` under `--rules`, and a repeated heading takes `-2`. A
section holding only a code block, an HTML comment, a table, a blockquote, an image or a link
reference is not a rule. A file with no heading, or with no section that is a rule, stays one
rule, named by its `rule:` key or else its file name, and its whole text - text above the first
heading included - binds the catalog as one unit, sentence by sentence as a section does:
measured when at least one sentence binds, unmeasured otherwise. Known misses: in a file with a rule section, text above the first
heading is no rule, a setext heading (text underlined with `===` or `---`) does not split,
and list items are never split.

**Detector validity is measured, and the measurement is small.** Every detector is scored
against a hand-labelled corpus that ships with the package - see
[How good are the detectors?](#how-good-are-the-detectors) below - but that corpus is
synthetic and it is ten sessions, so it catches a detector that is wrong about a shape it
was shown and says nothing about a shape nobody thought of. The detectors deliberately
under-count: a missed hit is a quieter report, a false hit is a wrong one.

**A tool call counts only when it ran.** A call the user refused or cancelled, one a
permission rule or policy denied, and one stopped by a check before the tool ran is not a tool
use for any detector, so a refused write of a key is no `secrets/secret-in-write` hit and a
refused edit opens no `testing/test-after-change` opportunity. A call that ran and failed
still counts: a test run that exits non-zero is a test run. Each runtime records the
difference its own way. Claude Code marks a refused call's result (`toolDenialKind`, or a fixed
refusal text in older versions, a PreToolUse hook's block included). Codex writes a fixed
output, `exec command rejected by user`, `patch rejected by user`, `aborted`, or
`apply_patch verification failed: ...` for a patch that never applied. Gemini CLI records a `status`, and only `success` counts.
Where a transcript cannot tell a call stopped part-way from one that never started, it is left
out: a call the user interrupted on Claude Code or Codex, and any Gemini call `cancelled` or
`error`. So an interrupted test run no longer counts as following a change: like a runner the
detector does not recognise, it can only lower `testing/test-after-change`'s followed share,
which is why that share is read as a floor. A refusal worded some other way is not recognised and still counts: a Codex exec policy's own
reason, and on Claude Code a hook's own deny reason on a line without `toolDenialKind`.

**Gemini CLI records less, so it is measured less.** Its sessions are read into the same
events, with `write_file` read as `Write`, `replace` as `Edit` and `run_shell_command` as
`Bash`, and six gaps under-count on it rather than guess:

- A context compaction is not told apart from truncation, tool-output masking and rollback,
  which rewrite the history the same way, so `cache-hygiene/compact` never fires on Gemini.
- The platform is not recorded. A shell call is `Bash` only when `.project_root`, in the
  project directory above `chats/`, holds a POSIX path. Every other root keeps the native
  name, so no `Bash` detector reads it: a drive letter or a UNC path, where the command is
  PowerShell, and a missing root, where the platform is unknown.
- The model changes by Gemini's router and quota fallback as well as by choice, and the
  transcript does not say which, so a Gemini event's model is left empty and
  `cache-hygiene/model-switch` never fires on Gemini.
- `invoke_agent` keeps its native name, so a detector on `Agent` does not see Gemini's
  subagent calls; each subagent's own session is read as a session of its own, with the id
  `<parent session id>/<its own id>`.
- Turns are not recorded. One starts at each user record that is not only tool responses.
- Edits the user made before accepting a write or an edit are not measured as writes. The
  call keeps its native name, `write_file` or `replace`, and its input drops `content`,
  `new_string`, `old_string` and `ai_proposed_content`, which may hold the user's text rather
  than the agent's. Its `file_path` and `modified_by_user` stay.

Every other shipped and catalog detector is scored on labelled Gemini sessions under a
POSIX root, near-misses included. A session under a Windows root checks that the shell
detectors stay silent there; it does not measure them on PowerShell.

Gemini deletes sessions older than 30 days by default, so a longer `--since` finds fewer.

**What a count is not.** A detector fires on a shape in a transcript, not on an intention.
`whole-file-cat` firing 89 times above does not prove the agent wasted context; it proves it
read 89 files whole, which is a fact worth having and an argument worth starting.

## Why did that fire?

A count you are surprised by is one you want to check. `ruleprobe explain` reruns the
detectors over the same transcripts and prints every hit `report` counts, with the event
behind it:

```sh
uvx ruleprobe explain                                   # every hit
uvx ruleprobe explain --detector verification/no-verify # one detector's
uvx ruleprobe explain --session claude-code:sess-1 --key 1:toolu_2
```

```
session   claude-code:sess-1
key       1:toolu_2
detector  verification/no-verify
turn      1
tool use  toolu_2 (Bash)
command   git commit --no-verify -m 'fix: the thing'
```

A hit's address is its session, `<runtime>:<session id>`, plus its key, `<turn>:<tool use id>`,
or `<turn>:-` for a hit that names no tool use - a compaction, a model switch - and so has no
single event to show, only its turn. A key is unique only within its session, so both are
printed. What is shown is the Bash command, or any other tool's whole input as JSON, because a
hit does not record which field of the event matched. `--session` takes the address or the bare
session id; `--session`, `--detector` and `--key` each narrow the output, and one that matches
nothing prints nothing and exits 0; `--detector` takes a renamed id too. It takes `report`'s
`--root`, `--runtime`, `--since`, `--stance` and `--plugins`, and `--rules`, `--detectors` and
`--no-config`, so a gated or plugin detector `report` counts is listed as well; what a detector
file skipped goes to stderr.

Every line it prints, to either stream, passes through `redact` first. It redacts AWS access
key ids, Bearer tokens, private keys (header to footer), Slack, GitHub, OpenAI and Anthropic
style tokens, a URL's password, and the value after a key name - `aws_secret_access_key`,
`client_secret`, and `password`, `passwd`, `pwd`, `token`, `api_key`, `apikey`, `secret` or
`access_key` followed by `:` or `=` - through the end of its line, its continuations, and the
indented lines below when the value starts there. Any other credential can still print, so
read explain's output before you share it. A control character or a Unicode format
character is printed as its `\xNN` or `\uNNNN` escape, so a transcript cannot drive your
terminal or reorder a line, and one event's text is cut at 4,000 characters with a note of
how many were cut. Like `report`, it writes nothing and
sends nothing, and running it changes no report number. A stored row keeps counts only, not
the events behind them, so it cannot be explained after the fact; an undeclared helper in
`ruleprobe.report` says so for a row and names the rerun over its session that would.

## How good are the detectors?

A hit rate is a rate of the detector until somebody says what the detector *should* have
found. So a labelled corpus ships inside the package, at `ruleprobe/corpus/`: ten synthetic
sessions in all three transcript shapes, every interesting event labelled by hand with the
detectors that ought to fire on it, and a deliberate near-miss beside each one - a `cat` of
a line range, a `find` narrowed by `-name`, a `git push` after the gate ran, a heredoc with
`rm -rf` in its body as text rather than as a command.

```sh
ruleprobe corpus
```

```
detector                                pos  neg   tp   fp   fn   prec  recall     f1  note
-------------------------------------------------------------------------------------------
cache-hygiene/compact                     5    6    5    0    0   1.00    1.00   1.00
cache-hygiene/model-switch                5   15    5    0    0   1.00    1.00   1.00
commits/non-conventional-subject          5    9    5    0    0   1.00    1.00   1.00
git-safety/force-push-default             5    7    5    0    0   1.00    1.00   1.00
package-manager/pip-install               5    7    5    0    0   1.00    1.00   1.00
secrets/secret-file-add                   5    6    5    0    0   1.00    1.00   1.00
secrets/secret-in-write                  16   22   16    0    0   1.00    1.00   1.00
testing/test-after-change                11    6   11    0    0   1.00    1.00   1.00
transcript-hygiene/unfiltered-find       10   15   10    0    0   1.00    1.00   1.00
transcript-hygiene/whole-file-cat        12   19   12    0    0   1.00    1.00   1.00
verification/no-verify                   11   13   11    0    0   1.00    1.00   1.00
-------------------------------------------------------------------------------------------
total                                    90  125   90    0    0   1.00    1.00   1.00  floor 0.90

binder over the rules zoo: 201 sections, 240 labels
catalog entry                           pos   tp   fp   fn   prec  recall  note
-------------------------------------------------------------------------------
cache-hygiene/compact                     4    3    0    1   1.00    0.75
cache-hygiene/model-switch                4    3    0    1   1.00    0.75
commits/non-conventional-subject          9    5    0    4   1.00    0.56
git-safety/force-push-default            13    9    0    4   1.00    0.69
package-manager/pip-install              10    8    0    2   1.00    0.80
secrets/secret-file-add                   9    4    0    5   1.00    0.44
secrets/secret-in-write                   4    4    0    0   1.00    1.00
testing/test-after-change                15   14    0    1   1.00    0.93
transcript-hygiene/unfiltered-find        3    3    0    0   1.00    1.00
transcript-hygiene/whole-file-cat         8    6    0    2   1.00    0.75
verification/no-verify                   11    3    0    8   1.00    0.27
-------------------------------------------------------------------------------
total                                    90   62    0   28   1.00    0.69  recall floor 0.68

binder over the independent held-out set: 80 sections, 80 labels
catalog entry                           pos   tp   fp   fn   prec  recall  note
-------------------------------------------------------------------------------
cache-hygiene/compact                     3    0    0    3      -    0.00
cache-hygiene/model-switch                3    1    0    2   1.00    0.33
commits/non-conventional-subject          4    1    0    3   1.00    0.25
git-safety/force-push-default             4    1    0    3   1.00    0.25
package-manager/pip-install               4    0    0    4      -    0.00
secrets/secret-file-add                   4    0    0    4      -    0.00
secrets/secret-in-write                   4    0    0    4      -    0.00
testing/test-after-change                 4    0    0    4      -    0.00
transcript-hygiene/unfiltered-find        3    1    0    2   1.00    0.33
transcript-hygiene/whole-file-cat         3    0    0    3      -    0.00
verification/no-verify                    4    1    0    3   1.00    0.25
-------------------------------------------------------------------------------
total                                    40    5    0   35   1.00    0.12  recall floor 0.12
```

The six shipped detectors and every catalog entry are scored over the corpus. A detector
the corpus labels is scored by those labels alone, so a catalog entry's own `examples:` score
its row only when the corpus labels none of it, and the row's `source` in `--json` then says
`examples`. The test suite runs every restating entry's examples against the shipped
detector, so they stay true of it.

`pos` and `neg` are what the labels asked for; `tp`, `fp` and `fn` are what happened.
`ruleprobe corpus --floor 0.9` exits non-zero when a scored detector falls under the floor,
and CI in this repository runs exactly that. It is a gate on the repository, not on a run:
nothing in `ruleprobe report` reads the floor, and no report of yours will ever fail because
a detector scored badly. `--json` prints the same numbers as data.

**A perfect score here is not field accuracy.** `secrets/secret-in-write` scored 1.00 on this
corpus while a hand check of 23 of its hits on real transcripts judged one right, 20 wrong and
two unsure, a precision of 0.05: the wrong hits wrote *about* secrets - a variable's name, a
pattern's source, a placeholder, redaction code. It now counts a live credential and never a
mention of one: an issuer's shape with a value behind it - an AWS access key id, a key whose
name ends in `aws_secret_access_key` or `client_secret` (`GOOGLE_CLIENT_SECRET` too) assigned a
literal, a Bearer, GitHub, Slack, OpenAI or Anthropic token, or a private-key header with a
body. A placeholder (`xxx`, `your-...`, `changeme`, `${VAR}`, a value naming itself a secret,
one repeated character, a prefixed token with no digit), a variable name or dotted path, a value
read from the environment, a line that reads as a pattern source or as redaction code
(`re.compile(`, a character class or quantifier, a `redact(` or `mask(` call), and, in a test
file, a token whose own line calls it fake count nothing; the corpus carries a near-miss for
each. A
documented example value in an issuer's shape still counts, since nothing in it tells it from a
live one, and a password still has no shape. Its field precision has not been retaken since the
change; the corpus score is the only figure for it.

The second table scores the rule binder, the text reading that binds a rule section to a
catalog entry, over `ruleprobe/corpus/rules-zoo.json`: synthetic rule sections with every
line labelled with the catalog detector it should bind, or none, beside near-misses for
exceptions, conditions and contrasts. `pos` counts the lines labelled with an entry, a heading
included, and a bind counts on the line its sentence starts on. Any false bind fails the command, whatever `--floor` says, because a rule bound to the wrong
detector is measured wrongly. On the shipped zoo, recall fails only under the recorded floor,
the recall the shipped binder measured, which rises as binding improves; a zoo of your own in
a `--corpus` directory reports its recall and holds it to nothing. Labels naming a detector no catalog
entry binds yet are counted apart. `--json` carries the same figures under `binding`.

The third table scores the binder the same way over `ruleprobe/corpus/rules-heldout.json`: 80
synthetic rule sentences, 40 of them labelled, written from each detector's claim by an author
who never saw the catalog's patterns, and shipped as written. Its recall, 0.12, is the honest
figure for rules the catalog was not written against: the catalog reads few of the ways people
phrase a rule, and leaves the rest unmeasured rather than guess. Any false bind fails the
command, and recall is held to its own floor on the shipped set's bytes. Once shipped it is a
regression set, not a fresh measure. `--json` carries it under `heldout`.

`ruleprobe report --validity` puts each detector's `p=` and `r=` beside its row. It is off
by default because the report is meant to be read in a minute and an eight-column table is
not, and because the same two numbers apply to every run - they belong to the detector, not
to your transcripts.

**Read the number for what it is.** The corpus is synthetic and hand-labelled: no real
transcript content, no home paths, no personal names. A 1.00 says the detector is right
about the shapes somebody thought to write down, which is a weaker claim than it looks - the
false positives a detector meets in the wild are the ones nobody anticipated. It is a floor
under an obvious mistake and a place to put the next surprising transcript, not a measured
field accuracy. Growing it is the cheapest contribution this repository takes: add a session
under `ruleprobe/corpus/sessions/`, label it in `ruleprobe/corpus/labels.yaml`, and the
table above moves.

**A wrong hit of yours becomes a labelled negative.** When `ruleprobe explain` shows a hit that
should not have fired, `ruleprobe label` records it in a corpus directory you name:

```sh
mkdir -p ~/ruleprobe-corpus
ruleprobe label --session claude-code:sess-1 --detector verification/no-verify \
                --key 1:toolu_2 --corpus ~/ruleprobe-corpus --name no-verify-in-a-message
ruleprobe corpus --corpus ~/ruleprobe-corpus
```

It reruns the detector over your transcripts, takes the one event behind the hit, and writes it,
redacted, as `sessions/<name>.events.jsonl` - one event per line, read back without a runtime
reader - with one `near` label for it in `labels.yaml`, which it creates or appends to so a
hand-written file keeps its comments. It is the only command that writes, and it writes nothing
outside that directory. It refuses, says why and writes nothing when the hit is a session hit
(`<turn>:-`), which one event cannot reproduce; when the written event no longer produces the
hit, because redaction changed what the detector matched or the hit needs the events around
it, since that negative would pass trivially; when a secret shape would survive into the
written bytes; and when the session file exists already or `labels.yaml` would not read back
as before plus one entry. It takes `report`'s `--root`, `--runtime` and `--since`, and
`--rules`, `--detectors` and `--no-config`; `--name` is a plain file stem. It takes no
`--stance`, because a detector gated on one is refused, since the corpus runs with none, and
no `--plugins`, because `ruleprobe corpus` loads none to score the negative with. A Claude Code
subagent's own transcript is its own session, keyed as `<parent session id>/<agent id>`, and a
transcript that yields no event is no session at all. Transcripts that share a session id, as
copies do, are one session in `report`, `explain` and `label`: the one with the most events is
read, the first in path order on a tie, and the rest are reported as copies set aside. When
the session read has no hit at the key and copies of it were set aside, the refusal names
them with their folders, so `--root` can point at the one that holds the hit. `ruleprobe
corpus` is the exception: a label names its session by file, so it reads each labelled file
as its own session, copies included. It prints the
`ruleprobe corpus` command that scores the new negative, with the same `--rules`, `--detectors`
and `--no-config` it was given. The new negative scores as a false positive until the detector
stops firing on it, which is the point.

In a corpus's `sessions/`, the suffix `.events.jsonl` is reserved: a file ending in it is read
as one event per line, never through a runtime reader, so a native transcript must not be
named that way.

**A detector of your own scores itself.** Rather than a corpus, a declarative detector may
carry an `examples:` block of minimal cases, and `ruleprobe corpus --rules ./docs/rules`
scores those:

```yaml
- id: house-style/sudo-install
  rule: house-style
  event: tool_use
  when:
    command: {starts_with: [sudo, pip]}
  examples:
    fire:
      - bash: sudo pip install ruff
    skip:
      - bash: uv pip install ruff
        note: the tool the rule asks for
```

`fire` is a list of cases the detector should fire on and `skip` a list it should not. A
case is `bash: <command>`, or `event: <one event>`, or `events: [...]` for a `session`
detector, with an optional `note`. Nothing runs them at report time. A detector with no
examples and no corpus label prints `no examples` rather than a number, and the floor steps
over it: an unmeasured detector is a gap to see, not a failure to fix.

## How it is put together

- `ruleprobe/events.py` - the event schema every reader emits: `assistant_text`, `tool_use`,
  `tool_result`, `user_prompt`, `compact`.
- `ruleprobe/readers/` - one module per runtime, turning a transcript into that schema.
  Claude Code, Codex and Gemini CLI today; a reader is `ROOT`, `transcripts()` and `read()`.
- `ruleprobe/shell.py` - the Bash decomposition every shell detector shares. Compounds,
  pipelines, heredocs, substitutions and continuations, parsed once per command.
- `ruleprobe/registry.py` - `Detector`, `Registry`, `run()`. Third-party detectors arrive
  through the `ruleprobe.detectors` entry point group or through `Registry.add`.
- `ruleprobe/declarative.py` - the YAML subset and the front-matter split, with a line
  number on every refusal.
- `ruleprobe/matchers.py` - one entry compiled into the same `Detector` a Python one builds.
- `ruleprobe/detectors/catalog.py` - the shipped catalog: common rule shapes as Python
  literals, each an anchored pattern and a declarative detector with its `examples:`.
- `ruleprobe/rules.py` - where detector files live, and which rule files nothing measures.
- `ruleprobe/report.py` - rows in, text out. A row is a small dict, so a report can be taken
  over rows you stored months ago rather than over transcripts you still have.

## The public API

The declared API is these names, at these import paths. `tests/test_contract.py` imports and
exercises every one, so removing a name or changing a call shape below fails the suite, and a
name joins the list only with its test. Every other name in `ruleprobe.__all__` is importable
but not declared, so it may change in any release.

- `ruleprobe`: `Registry`, `analyse`, `compile_detector`, `counts`, `is_undecided`,
  `iter_sessions`, `load_bundle`, `report`, `report_data`, `run`, `validity`
- `ruleprobe.declarative`: `load`
- `ruleprobe.detectors.common`: `DETECTORS`, `SECRET_PATTERNS`
- `ruleprobe.events`: `hit`, `input_of`, `text_of`
- `ruleprobe.registry`: `Detector`, `Registry`
- `ruleprobe.shell`: `Context`, `MARKER_RE`, `MAX_COMMAND`, `Parsed`, `SUB_PLACEHOLDER`,
  `git_calls`, `has_redirect`, `normalise`, `operands`, `pipelines`, `strip_heredocs`
- `ruleprobe.validity`: `CorpusError`, `DEFAULT_FLOOR`, `Score`, `below_floor`, `score_corpus`,
  `scores_as_dict`, `validity_table`

Measuring and reporting:

```python
iter_sessions(root=None, runtime="auto", since=None, errors=None)  # -> Session(.id .repo .events)
                                          # one per runtime and id; errors also holds each copy
                                          # set aside: error == readers.COPY, with kept
run(events, stances=None, *, registry=DEFAULT, strict=False, errors=None)
                                          # -> {detector_id: [Hit(id, turn, tool_use_id)]},
                                          # detectors with no hit omitted
Registry([Detector, ...]); Registry.add(Detector(id, rule, event, fn, gate=None))
Registry.from_entry_points("ruleprobe.detectors")
report(rows, by="rule", min_sessions=20, frequent_share=0.30, registry=DEFAULT,
       validity=None, min_opportunities=20)
report_data(rows, by="rule", ...)                       # the same numbers as a dict; --json prints it
validity(registry=DEFAULT, directory=None)              # -> {detector_id: Score(.precision .recall)}
load_bundle(paths=None, rules_dir=None, cwd=None, config=True)  # -> Bundle(.detectors .rules .findings)
compile_detector(spec, path="<spec>", lines=None)               # -> Detector
```

Writing a detector in Python:

```python
Detector(id, rule, event, fn, gate)       # positional, gate optional; a subclass may add
                                          # __slots__ = () and properties named kind, stance
fn(events, ctx)                           # -> [(turn, tool_use_id), ...]
Detector(..., opportunities=count)        # keyword only; count(events, ctx) ->
                                          # [(turn, tool_use_id, followed), ...], followed
                                          # True, False or None when undecided
ctx.events, ctx.bash, ctx.finals          # a Context: every event, each Bash call as a
                                          # Parsed(.event .command .heredocs .skipped
                                          # .pipelines), final messages
hit(event); hit(event, tool_use_id=False) # -> (turn, tool_use_id or None)
git_calls(parsed, ("commit",))            # yields (segment, subcommand, args)
MARKER_RE.match(token).group(1)           # an index into parsed.heredocs
input_of(event); text_of(value); normalise(command)
pipelines(command)                        # -> [[[token, ...], ...], ...]; [] on an
                                          # unterminated quote
operands(segment); has_redirect(segment); strip_heredocs(command)  # -> (text, bodies)
```

A `gate` is `None` or a `(dimension, variants_or_None)` pair. A detector's `event` is one of
`bash`, `write`, `agent-brief`, `assistant-final`, `session`, `tool_use` or `assistant_text`;
a command longer than `MAX_COMMAND` is `skipped` and never tokenized. A detector reads the
event fields `kind`, `turn`, `id`, `name`, `input`, `text`, `final`, `tool_use_id` and
`tool_name`, each on the kinds `ruleprobe/events.py` documents it for.

Scoring against the corpus:

```python
score_corpus(registry=DEFAULT, directory=None)  # -> {detector_id: Score}; raises CorpusError
Score(detector_id)                              # .detector .scored .precision .recall .add(other)
below_floor(scores, floor); scores_as_dict(scores, floor); validity_table(scores, floor)
declarative.load(path)                          # -> (document, lines)
```

A predicate from `compile_matcher` returns true, false, or a falsy undecided value when it
could not read its input. A Python caller who negates one itself reads undecided as false and
can over-count: compose through the declarative `not`, `any` and `all`, or test the result
with `is_undecided(value)` before negating it. `compile_matcher` itself is not declared: it
needs arguments the package does not declare, so it is not yet a declared way to build a
predicate, and `is_undecided` is declared ahead of it.

These promises are not names:

- `SECRET_PATTERNS` stays a plain module-level assignment of a literal list in
  `ruleprobe/detectors/common.py`, so it can be read by syntax tree without importing, from
  the source tree or from that member of the wheel. Every pattern compiles with `re`.
- `DETECTORS` in `ruleprobe.detectors.common` is exactly the six generic detectors:
  `cache-hygiene/compact`, `cache-hygiene/model-switch`, `secrets/secret-in-write`,
  `transcript-hygiene/unfiltered-find`, `transcript-hygiene/whole-file-cat` and
  `verification/no-verify`.
- The wheel is pure Python and named `ruleprobe-<version>-py3-none-any.whl`.
- The package imports and runs from that wheel placed on `sys.path` as a zip, uninstalled.
- The wheel carries the corpus at `ruleprobe/corpus/`, beside `ruleprobe.__file__`. A caller
  importing from a zip unpacks it, or points `RULEPROBE_CORPUS` at a copy.
- Its `labels.yaml` writes every `fire` and `near` as a flow list, so a caller can read the
  labelled ids by pattern. `score_corpus(directory=)` scores a caller's own corpus in the
  same layout, ignoring a top-level key it does not read.

## Versioning

0.2.0 versions the contract: the declared API above, the row schema (`schema_version` on each
row and on the `report_data` result) and the detector-entry schema (`schema_version` on an entry,
`version` on a detectors file). [CHANGELOG.md](CHANGELOG.md) lists every part of that break and
its migration.

- **Within a minor series, nothing declared breaks.** No declared name, call shape, row schema
  or entry schema changes incompatibly between 0.2.0 and any 0.2.x. A patch may add; it may not
  remove or reshape. A detector renamed in a patch ships its fold entry, so rows stored under
  the old id still count under the new one, and a test fails when a shipped id is neither
  registered nor folded. The release preflight refuses a patch release whose contract test no
  longer declares a name its series' `.0` tag declared.
- **A later 0.x minor may break the declared surface, with notice.** Its changelog section opens
  with a **Breaking** heading naming the migration, and the contract test changes in the same
  release. A tool pinning a minor should treat each minor bump as a possible break: read that
  heading and re-run its own contract tests before bumping.
- **From 1.0, a break is a major.**

## Development

```sh
git clone https://github.com/JakeSelby/ruleprobe && cd ruleprobe
python3 -m unittest discover -s tests
uv run --python 3.9 python -m unittest discover -s tests   # the floor the package claims
python3 -m compileall ruleprobe
python3 -m ruleprobe report --root tests/fixtures
python3 -m ruleprobe report --root docs --rules docs/rules
python3 -m ruleprobe corpus --floor 0.9                    # the corpus gate CI runs
```

## Contributing and releases

Issues, false positives and labelled corpus sessions are welcome; [CONTRIBUTING.md](CONTRIBUTING.md)
says how a change lands, and [docs/releasing.md](docs/releasing.md) how a version ships. Questions go
to [Discussions](https://github.com/JakeSelby/ruleprobe/discussions).

## Origins and neighbours

The engine was carved out of [Model Citizen (formerly agent-harness)](https://github.com/JakeSelby/model-citizen),
where it grew as a hook that measured that project's own always-loaded rules; the detectors
that were about agent-harness's rules stayed there, and the rule-agnostic half is this
package. The nearest neighbour is [Burnd](https://github.com/garvitsurana271/burnd), which
also reads Claude Code transcripts locally, for token spend rather than for rule compliance.

MIT licensed.
