Metadata-Version: 2.4
Name: my-mend
Version: 0.12.0
Summary: myMend repairs OCR'd books: a PostgreSQL-backed document pipeline, with rinse (raw OCR to editions) and clean (a regex pattern library) as its deterministic tools.
Author: Robb Doering
Author-email: Robb Doering <robb@doering.ai>
License-Expression: MPL-2.0
License-File: LICENSE
Classifier: Development Status :: 4 - Beta
Classifier: Programming Language :: Python :: 3.13
Classifier: Framework :: Pydantic :: 2
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Environment :: Console
Classifier: Typing :: Typed
Requires-Dist: jinja2>=3.1,<4
Requires-Dist: pydantic>=2.11,<3
Requires-Dist: sqlalchemy>=2.0.44,<3
Requires-Dist: psycopg[binary]>=3.2,<4
Requires-Dist: intact>=1.2.1
Requires-Dist: markdown-it-py>=4.2.0
Requires-Dist: mdit-py-plugins>=0.6.1
Requires-Dist: my-basis>=1.2.4,<2
Requires-Dist: myform>=1.2.0,<2
Requires-Dist: regex>=2025.11.3
Requires-Python: >=3.13
Project-URL: Homepage, https://gitlab.com/doering-ai/libs/mend
Project-URL: Source, https://gitlab.com/doering-ai/libs/mend
Project-URL: Issues, https://gitlab.com/doering-ai/libs/mend/-/issues
Description-Content-Type: text/markdown

<!-- readme-header:begin -->

<div align="center">

# myMend: _Agent-Friendly Repair for OCR'd Books_

<p align="center">
  <img src="https://img.shields.io/gitlab/pipeline-status/doering-ai/libs/mend?branch=main" alt="pipeline status"> <img src="https://img.shields.io/gitlab/pipeline-coverage/doering-ai/libs/mend?branch=main" alt="test coverage"> <a href="LICENSE"><img src="https://img.shields.io/gitlab/license/doering-ai/libs/mend" alt="license"></a> <a href="https://gitlab.com/doering-ai/corpus/-/blob/main/policies/versioning.md"><img src="https://img.shields.io/badge/version-0.12.0-blue" alt="version"></a>
</p>

<p align="center">
  <a href="https://github.com/pre-commit/pre-commit"><img src="https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit" alt="pre-commit"></a> <a href="https://github.com/facebook/pyrefly"><img src="https://img.shields.io/endpoint?url=https://pyrefly.org/badge.json" alt="pyrefly"></a> <a href="https://github.com/astral-sh/ruff"><img src="https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json" alt="ruff"></a>
</p>

</div>

<!-- readme-header:end -->

myMend repairs OCR'd books. `mend.pipeline` is the authority: it keeps complete document revisions in PostgreSQL and runs explicit Actions that change them. `rinse` and `clean` are the deterministic tools around it: `rinse` builds an edition from raw OCR, and the pipeline's cleaning Action runs `clean`'s rules. Nothing in the package calls a model. INTAKE's eight verbs on the Store are in `docs/intake.md`, PROOF's generic runner in `docs/proof.md`, and SQL-backed work-level reading in [`docs/native-proof.md`](docs/native-proof.md).

<blockquote class="artificial-prose">

*Left-rule blocks mark artificial prose; quotations retain their own attribution.*

</blockquote>

Repair history can also develop executable rules through finite, resumable batches. The [symbolic cleanup guide](docs/symbolic-cleanup.md) connects preparation, learning, qualification, independent feedback and public CLEAN application. Search checkpoints preserve completed trials and costs; learned artifacts remain explicit, with their qualification and source-evidence grades.

## Installation

<blockquote class="artificial-prose">

Python 3.13 or later is required.

From a checkout, in a virtual environment:

</blockquote>

```console
$ uv pip install .
# or
$ python -m pip install .
```

## Commands

<blockquote class="artificial-prose">

The package installs three console scripts. `mend-pipeline` is the pipeline and the one to learn first; `docs/pipeline.md` is its manual. `rinse` and `clean` are the deterministic tools around it (a cleaning Action runs `clean`'s rules), and they stay usable on their own. None of them calls a model.

</blockquote>

| Command         | Purpose                                                                            | State   |
| --------------- | ---------------------------------------------------------------------------------- | ------- |
| `mend-pipeline` | Keep document revisions, run Actions, export and apply edits (`docs/pipeline.md`). | working |
| `rinse`         | Turn raw OCR output into an edition, structure first, then sembr.                  | working |
| `clean`         | Run the pattern library over Markdown, in place; a second run is a no-op.          | working |

```console
$ clean works/                       # repair in place
$ clean --check works/               # write nothing; exit 1 if anything would change
$ clean --diff works/hegel/chapter-003.md
$ clean --check --report edits.jsonl works/   # every edit and detector hit, with context
$ clean --no-corpus --summary convergence.jsonl draft/ # isolated context; explicit fixed-point receipt
$ clean --check -j 12 --rule spacing works/   # one concern (or `concern.rule`), 12 works at a time
$ clean --check --corpus ~/my/lit raw/book.md  # weigh rare tokens against all of lit's English text
$ clean --family legal --skip legal.page_footer docs/   # a document family; leave one rule out
$ clean --list-rules --family legal              # what the family runs, rule by rule
```

## Rinse

<blockquote class="artificial-prose">

`rinse` turns the raw OCR of a book into a lit edition, structure first. It reads a work's source (`replay/sources/<hash>.md`, read only) and, when the raw shelf holds a byte-identical copy, Marker's `.metadata.json` beside it. It writes `index.md` and `chapter-NNN.md` and nothing else: the pictures under `assets/` and at the root of an edition are never moved or deleted, and only stale `chapter-*.md` files are removed. The build is deterministic and idempotent: a second run changes nothing. It makes no model call.

</blockquote>

<blockquote class="artificial-prose">

`--intake` takes a candidate ledger in the schema of lit's retired `intake.yaml`, which lit deleted on 2026-10-19 (MEMY-2113). The `rinse` Action on the Store replaces that admission (`docs/intake.md`, section 4); until then the command runs only against an explicit ledger file.

</blockquote>

```console
$ rinse hegel-g__1807__phenomenology-of-spirit --intake intake.yaml   # rebuild a lit work in place
$ rinse --check --diff WORK... --intake intake.yaml   # write nothing; print the diff; exit 1 if any edition would change
$ rinse WORK... --intake intake.yaml --out /tmp/sample   # write /tmp/sample/<slug>/ and leave lit alone
$ rinse --source book.md --intake intake.yaml --out ~/ai/scratch/new   # admit a new source by the ledger
```

<blockquote class="artificial-prose">

Exit codes follow `clean`: 0 when nothing is wrong, 1 when `--check` or `--diff` finds an edition that would change, 2 when a work could not be rinsed.

`--intake FILE` is required. It names lit's generated `lit.intake.v1` ledger, which is the only admission policy. A new source must match both the recorded path and full lowercase SHA-256, have status `verified` or `promoted`, and have a supported tier (`core`, `primary`, `secondary`, `tertiary`, `auxiliary`, `nonacademic`, or `engineering`). The ledger supplies the slug, title, author and year, and a source's own YAML header never overrides them. Output goes to `--out/<slug>/`. Relative ledger paths resolve beside the ledger. Candidate, held and withdrawn rows may retain null hashes for missing evidence with an explicit `verdicts.reason`; verified and promoted rows require full hashes. `source.metadata` may name a Marker metadata file; otherwise the source's `.metadata.json` sibling is used, only while the recorded source bytes match. Missing or malformed ledgers, duplicate slugs/source paths/source hashes, held or withdrawn candidates, unknown tiers and changed bytes fail with exit 2 before an edition is written. `--json` includes the refusal in an `error` field. The ledger reader does not scan the shelf or fall back to ai.

An edition is `index.md` (the curated front matter kept key for key, a title, a byline and a `toctree`), `chapter-000.md` (`# [Front matter]`, each front item an H2, the printed contents kept as a nested list of titles), and one `chapter-NNN.md` per top-level section of the work, cut at H2 boundaries only above about 250 KB. Square brackets mark a heading rinse added (`[Front matter]`, `[Text]` for a work with no headings, `[Title, continued]`); no other heading is ever invented. The back-of-book index is dropped, and it ends where the index does: at the heading of the notes behind it, or of the errata, abbreviations, bibliography or books referred to that follow it, which stay with their pictures. An index set in long paragraphs of entries is still an index: a paragraph in which a tenth of the words are page numbers or ranges (`223,` `290-291;` `xxix-xxxi;`) is no running text, and a headword with no number of its own (`Greece`) does not end an index whose entries follow it. A line that opens with a note number is a note, never a running head, however often its words recur. Inherited nonempty image alt survives an empty or missing target with a visible limitation; available images retain access text separately from printed captions (see [image access](docs/rinse-images.md)). `% page N` comments mark the first block of each PDF page the source lets rinse place, strictly increasing, never guessed.

</blockquote>

| Step (`mend/rinse/`) | What it does                                                                                                                                                                                                                                                                                                              |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `html`, `markup`     | Marker's HTML to MyST; `{sup}'12'` roles mended; a superscript `a`, `I` or `A` read as the letter (by shape only, never a Bekker mark); padded table cells trimmed; links to gone anchors unlinked; the linked notes of a Project Gutenberg HTML edition become `{sup}` roles and note paragraphs, which the binder takes |
| `pages` (`layer`)    | A text layer's `Page N` paragraphs become page labels; its `# Pages 001-020` headings and `[No embedded text on this page.]` placeholders go                                                                                                                                                                              |
| `figures`            | Local images become native figures with alt, separate printed captions, and linked long descriptions. Missing images retain useful alt with a visible limitation. Images in prose blocks are lifted after their containing block; code and math remain literal. See [image access](docs/rinse-images.md).                 |
| `pages`, `marker`    | Anchors, `_page_N_` image names and the headings of Marker's table of contents become `% page N`; only the first of several PDFs is labelled                                                                                                                                                                              |
| `furniture`          | Running heads (recurring, set like a title, cutting sentences), printed page numbers, scan filler, extraction comments (`<!-- source-pdf-sha256 -->`) and repeated comment stubs (`Note -->`)                                                                                                                             |
| `matter`             | Gutenberg's wrapper, the back-of-book index (not the notes, errata or abbreviations behind it, and not an `Index` among the entries of a printed contents), and the cut between front matter and work                                                                                                                     |
| `joins`, `lists`     | A paragraph cut by a page break is whole again, and so is a note cut at the foot of a page whose end the scan marks with a rule (the rest of the note opens the next page, in lower case); `1) a 2) b 3) c` and bullet runs become lists                                                                                  |
| `plain`              | A text with no headings of its own: the printed contents and division labels (`SECTION II.`) give the headings; nothing is guessed from capitals                                                                                                                                                                          |
| `headings`           | Which blocks are headings, and how deep (below)                                                                                                                                                                                                                                                                           |
| `layout`, `sembr`    | Chapter files (an essay and the notes that close it are one file, the notes an H2); one sentence to a line (myform), applied to a paragraph only if its non-whitespace characters are exactly what they were                                                                                                              |

### Heading inference

<blockquote class="artificial-prose">

Marker's ATX levels are noisy (a book's chapters may come out at `#`, `##` and `###` by turns), so rinse reads each heading's numbering (`Chapter IV`, `II.`, `b.`, `3.2.1`, `§ 12`, `(AA.)`, `Teil 2`) as a set of candidate readings and searches, over the whole sequence, for the nesting that breaks the fewest habits of books: a number that continues its series is a sibling, one that restarts it (`A.` after `IV.`) opens a deeper level, `Part` outranks `Chapter`, a dotted number sits under its parent, and an unnumbered `Proof` or `Remark` is a leaf. Marker's own level breaks ties. A last pass puts the headings that follow a `Volume`, `Part` or `Book` inside it, up to the next one of its size or the notes and bibliography behind the work, whatever level Marker gave them; the `Part`s of a series that runs through the whole work (no place comes twice) are raised when Marker set them below their chapters, and a `Part` and a `Book` do not hold one another, since works put either first. A heading is never added except from evidence: a short line that Marker's table of contents lists, a line that is nothing but `Preface` or `CHAPTER 3`, or (in a source with no headings) a title the printed contents lists. Paragraph-length "headings", running heads, contents and index entries are turned back into text. Depth runs to six.

Whatever rinse decides that a rerun must repeat is written under one `rinse:` key of `index.md`: `body_start` (the heading that opens the work), `demote` and `promote` (titles that are, or are not, headings), `levels` (a title's forced depth) and `keep_sup` (letters whose superscripts are notation in this work). A curator edits that key; rinse reads it and never overwrites it.

</blockquote>

## The pattern library

The [CLEAN families guide](docs/clean-families.md) groups the current repairs, explains existing note support and distinguishes lexical guards from embeddings or POS tagging.

<blockquote class="artificial-prose">

`clean` runs contextual OCR rules. Its actual output then uses the shared [canonical reading policy](docs/text.md): ordinary searchable accents, quotes, fraction characters, ellipses, dashes, ligatures and spaces. The policy is independent of language guessing and family selection; `--rule` and `--skip` select contextual repairs while canonical output still applies. Human-readable bibliographic fields and editorial access text follow the same policy.

A rule names one pattern in a concern's `RegexStore` and says what becomes of each match: a template string, a *niche* function `(match, context) -> str` for decisions a template cannot make, or nothing, which makes it a *detector* that reports hits for reviewers. Rules run in order, one whole-text pass each, and never touch a *fence*: front matter, code, math, inline code (idx tags), roles, comments (page labels), HTML tags, link destinations, footnote labels and URLs (`mend/fences.py`). A niche function sees the whole work, not just its file, so a word split in one chapter can be rejoined as another chapter spells it. A rule that is right in some languages only names them (`languages=('en',)`), and the rulebook skips it for a work in another: a work's language is guessed from its function words (`mend/language.py`), and a short text that cannot be told counts as `other`.

Two more gates keep a rule to the text it was written for. A rule marked `prose=True` runs only in English running prose, judged by the share of function words, so a table, a bibliography or a page of notation is left whole. A rule marked `corpus=True` weighs a token against the whole library: `Context.corpus` counts every token of every English chapter in `works/` (`mend/corpus.py`, cached under `$XDG_CACHE_HOME/mend`), because damage is rare and the word it hides is common (`rnight` once, `might` ten thousand times). Without a corpus such a rule makes no edit. The lexicon (`mend/lexicon.py`, `mend/data/english.txt`) answers the other question, whether a string is a word at all.

For its reviewed default rulebook, the CLI keeps its newly loaded corpus counts in a read-only snapshot shared across the run. Custom rulebooks and caller-supplied corpora keep their existing live mapping behavior.

A repair can make another one visible, so `clean` repeats the rulebook over a work until a round leaves final canonical text unchanged (five rounds at most), and rebuilds the evidence between rounds. A final round without a text change establishes convergence, including when it is the fifth round. Text still changing at the cap means convergence was not established: the command reports cap exhaustion and exits 2. In ordinary writing mode it still writes the capped result; callers must inspect the exit code before accepting it. One limit remains: cleaning lit itself changes the counts the corpus gives, so a first run over lit can leave a few dozen splits that a second run finds, and a third finds none.

`--summary FILE` writes one JSON-lines convergence record per selected file, even when nothing changes: `file`, `kind: "summary"`, `changed`, `rounds`, `converged` and `final_round_edits`. Convergence and the final edit count describe the whole work because sibling files supply evidence. This is separate from the unchanged `--report` edit/hit format. Summary and edit reports must be distinct files outside cleaning inputs. `clean` writes by default; `--check` and `--diff` leave inputs intact. `--check` exits 1 on changes; legacy `--diff` alone exits 0 even when it prints changes, so an empty-diff check must require both success and empty stdout. Cap exhaustion exits 2 in either mode.

`--no-corpus` explicitly skips corpus token context and catalog title context, including automatic ancestor discovery. Rules requiring that context make no corresponding edits; diagnostics name skipped context. Work sibling evidence still runs. Record this selection with a preparation recipe rather than treating skipped context as a passed check. The option is mutually exclusive with `--corpus ROOT`; ordinary discovery defaults remain unchanged.

</blockquote>

| Concern (`mend/stores/`) | Rules                                                                                                                                                                                                                                                                          |
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `glyphs`                 | `glyph`: ligatures and invisible characters, from the table `mend/data/glyphs.yaml`; `control_split`; detectors `replacement`, `mojibake`, `control`                                                                                                                           |
| `markup`                 | `entity` (HTML escapes, from `mend/data/entities.yaml`), `duplicate_bullet`, `stray_grave`                                                                                                                                                                                     |
| `prep`                   | `title` (reviewed opt-in catalog-title styling), `two_dots` and `mixed_dots` (a damaged ellipsis set as the work prints it), detector `greek`                                                                                                                                  |
| `source`                 | `exact` (finite source-proved OCR readings with structural guards); detector `attached_link` (ambiguous attached local link or function notation)                                                                                                                              |
| `front`                  | `contact_line`: the e-mail and corresponding-author lines of an article's first page                                                                                                                                                                                           |
| `spacing`                | `double_space`, `space_comma`                                                                                                                                                                                                                                                  |
| `hyphen`                 | `broken_word` (rejoins `philo- sophy` as the work spells it), `line_break`, `paragraph_break`, `dot_break`; detector `unresolved`                                                                                                                                              |
| `numbers`                | `note_marker`; detectors `fused_word`, `stray_marker` (a paragraph number whose pilcrow the scan lost, among pilcrow-numbered paragraphs)                                                                                                                                      |
| `words`                  | `ligature_space`, `misread` (table `mend/data/misreads.yaml`), the corpus rules `ocr_swap`, `split_token`, `split_initial`, `joined_function`, `letter_spaced`, `ligature`; detectors `letter_run` (letter-spaced type left unjoined) and `spaced_capital` (`T HE`, `N OTE S`) |
| `dashes`                 | `double_hyphen`                                                                                                                                                                                                                                                                |
| `punctuation`            | `space_stop`, `space_colon`, `double_comma`, `double_period`, `glued_stop`, `stray_dot`; detectors `middle_dot`, `comma_stop`                                                                                                                                                  |
| `quotes`                 | `adjacent`, `apostrophe_space` (bounded English possessives and contractions), `low_pair` (German low quotes)                                                                                                                                                                  |
| `breaks`                 | detectors `split_paragraph` (a sentence across a paragraph break) and `lone_number`                                                                                                                                                                                            |
| `legal`                  | the `legal` family only: `redaction_bar`, `disclaimer`, `page_footer`, `bates_line` (published stage), `filed_stamp`, `received_stamp`; detectors `digit_line`, `confidential_stamp`, `privileged_stamp`, `exhibit_label`, `foia_stamp`                                        |

`clean --rule prep --corpus /path/to/lit` repairs mixed existing ellipses and reports Greek. Inferring an omission from two dots requires reviewed opt-in (`--rule prep.two_dots --force`); a dominant ellipsis style alone does not prove that the dots denote an omission. Reviewed title styling requires `clean --rule prep.title --force --corpus /path/to/lit`; catalog membership alone does not establish a quoted title's style. The catalog supplies full titles and pre-colon short forms; capital-started quoted matches become italics, except common one-word titles, nested quotations and existing emphasis. Missing or malformed catalogs make no title edit. Damaged ellipses use the unique dominant intact form in the work's unprotected text; ties, missing evidence and a lone adjacent sentence stop stay for the reader. Greek words are reported without editing; near-miss catalog titles still need a reader.

`source.exact` uses the finite readings and scope policy in `mend/data/source_repairs.yaml`. Missing structural evidence refuses a repair; code and markup cannot supply corroboration. This is not general running-head removal or junk-suffix stripping. `source.attached_link` reports forms such as `DT[theorem](problem)` without escaping them: a deliberately attached local Markdown link can have the same syntax. Use its report for source judgment; it is not a strict-build gate.

Frequency-based spelling repair preserves all-uppercase technical acronyms, named literals, apostrophes/primes and known lexicon words. Diagram separators remain intact. The reviewed compound policy in `mend/data/lexicon.yaml` keeps `well-known` when the work supplies no explicit closed spelling; rebuilding the dictionary retains that exclusion.

### Document families

<blockquote class="artificial-prose">

One library repairs more than one kind of document. A *family* is a profile of the rules (`mend/data/families.yaml`), not a fork: for each family, the rules it runs, the rules it only reports (a repair listed under `detect` runs as a detector), and the rules it switches off. `--family NAME` picks one; the default is `academic`, an explicit profile that leaves catalog-title styling and frequency-based OCR substitutions off until reviewed opt-in (`--rule prep.title --force` or `--rule words.ocr_swap --force`). Frequency alone cannot establish the intended word: valid rare words, page continuations and numbered references can resemble more common words. Exact misread and source-proved repairs remain enabled. The default and explicit family still agree (`tests/test_families.py` holds that against the existing cases and frozen long texts). `legal` is for court papers, transcripts, mail and records: the 21 rules that the APPS-497 pilot found safe on 398 such documents, no rule that reads lit, and the `legal` concern, which adds a rule only this family has (`Rule.families`). A family places every rule of its library exactly once, so a rule added later fails the tests until someone decides where it goes.

The academic profile also runs `words.split_lexical` for guarded split-word repair. It protects explicit spelling/specimen/identifier contexts, code, math and meaningful `differance` distinctions; ordinary quoted prose remains eligible. Use `--skip words.split_lexical` to omit it. Unique repeated same-work lost-letter recovery remains available explicitly through `--rule glyphs.work_diacritic --force` or `--rule glyphs.work_ascii --force`; the legal profile keeps all three off. Canonical accent folding still applies to resulting prose.

`--skip ID` leaves out a rule or a concern, and may be repeated. `--rule` stays include-only, and in a profiled family it may name only rules the family runs; `--force-rule` lets it name a rule the family has switched off, and run it as the library registers it. `--list-rules` prints what the family does with each rule and exits. `--stage published` is for text that has already been split into pages: a rule limited to a stage (`Rule.stages`) runs only there, and the default stage is `source`. The family, skip list and stage reach every worker of `-j N`.

The `legal` concern ports ten of the twelve accepted `epstein_legal` fixes. It repairs a redaction bar set as block characters (`[REDACTED]`), a `Page X of Y` line and, at the `published` stage only, a bare `EFTA` Bates line; it reports a bare run of six digits or more and the stamps `CONFIDENTIAL`, `PRIVILEGED`, `EXHIBIT` and `RELEASED PURSUANT TO FOIA`; and it holds two repairs, for `FILED:` and `RECEIVED:` date lines, off until a corpus shows them. The disclaimer that a mail system appends to a page is a bounded repair (`legal.disclaimer`): it removes the notice, from `please note` at the head of a line to its last sentence or, when the page broke inside it, to the Bates stamp that follows, and never the stamp, a signature or any text after it. A repair that removes a line leaves the line break behind, as the original fixes did, except `page_footer` and `legal.disclaimer`: when one of them takes a whole line it takes the line break too, so that a paragraph cut by a page break is whole again, and when text or a stamp shares the line it leaves the line as it was. The legal profile leaves `legal.disclaimer` off, with four of the pilot's repairs (`glyphs.glyph`, `punctuation.double_period`, `punctuation.space_colon` and `punctuation.space_stop`), because a review of 40 sampled edits per rule found more than one wrong edit in each (D-22); `--force-rule` still runs one, as the library registers it, and the academic family is unchanged. It runs one hyphen join, `hyphen.line_break`, which the pilot read at 6 of 26 wrong and which a fresh review of its edits decides (D-26); `hyphen.paragraph_break`, `hyphen.broken_word` and `hyphen.dot_break` stay off. The cases are `tests/examples/clean.legal.yaml`.

</blockquote>

```console
$ clean --family legal --check --report edits.jsonl docs/
$ clean --family legal --stage published pages/       # also removes bare EFTA lines
$ clean --family legal --rule hyphen.broken_word --force-rule note.md   # a rule the family switched off
```

### Adding a rule

<blockquote class="artificial-prose">

The library grows by turning accepted proofreading experience into tested symbolic capability. [The operator's vision](VISION.md) explains the purpose; [the learning specification](LEARNING.md) defines finite learning cycles, narrowly scoped cases, uncertainty-based review and the evaluation needed to measure useful growth. [The source-scoped learned-case contract](LEARNED-CASES.md) defines explicit source/member applicability, stable case attribution and Store-backed identity propagation. Standalone cleaning with unknown identity leaves those cases inactive. Contributors should be able to extend another agent's work without reconstructing its session. The tests hold every rule to the same contract.

1. Put the pattern in the concern's store (`mend/stores/<concern>.py`), named, using the `regex` module's syntax through `RegexStore` (subroutines with `(?P>name)`, atomic alternations with the `'<|>'` mark); never stdlib `re`. A new concern is a new module with `NAME`, `STORE` and `RULES`, listed in `mend/stores/__init__.py`.
2. Add a `Rule` to the module's `RULES`: a template, a niche function, or no replacement for a detector; add `languages=` when the repair is right in some languages only, `prose=True` when it is right in running English prose only, `corpus=True` when it weighs a token against all of lit, `encloses=True` when its match may take whole fences with it, and `families=('name',)` when the rule belongs to one document family only (a concern of its own, listed under that family in `mend/data/families.yaml`; `stages=('published',)` limits it to a stage). Literal swaps belong in a YAML table under `mend/data/`.
3. Add pattern cases to `tests/examples/regexes.yaml` (what matches, what must not).
4. Add rule cases to `tests/examples/clean.yaml` (`clean.legal.yaml` for a rule only the `legal` family has): at least one input the rule changes (or reports) and one it must leave exactly alone. A case may carry `lang: en` to fix the language a short text cannot show, and `lit: {token: uses}` to give a corpus rule the counts it weighs (every other token of the text counts as common). Every case also runs twice; the second pass must change nothing.
5. Run `task test -- tests/test_clean_snapshots.py`. It compares frozen long texts, work evidence, edits, and detector hits. Read every changed passage before accepting an intentional update with `task test:ssu -- tests/test_clean_snapshots.py`, then inspect `git diff -- tests/__snapshots__/test_clean_snapshots.ambr`. Unexpected changes become negative cases. See `tests/fixtures/clean/README.md` for provenance and the update contract.
6. Run `clean --check --report edits.jsonl --rule <your rule>` over lit's `works/` and over the raw sources in `replay/sources/` (lit holds what was already repaired, the sources hold the damage), read a sample of the rule's edits yourself. Read 40 sampled edits per rule, with a seed you record. A repair lands when at most one of the 40 reads wrong and you have looked at that one. A misfire becomes a leave-alone case, and so does any clean edit a reviewer later reverses.

Outside the library the same rule holds: a module that needs a pattern keeps it in a `RegexStore` called `STORE` beside its code, with a `NAME`, and lists the module in `mend/stores/support.py` (rinse's modules in `mend/rinse/stores.py`), so that `tests/test_support_regex.py` asks for cases in `tests/examples/support_regexes.yaml`. Write a space or a word as the module's old engine read it when a pattern moves from stdlib `re` (`RE_SPACE` and `RE_WORD` in `mend/stores/common.py`), and hold the move to the old pattern in `tests/test_legacy_regex.py`. The one pattern that stays raw is `smoke`'s bytes pattern for PDF streams: a `RegexStore` composes `str` only.

</blockquote>

## Downstream pins

<blockquote class="artificial-prose">

Corpus pins this package by git revision and moves its pin deliberately. `apps/epstein-index` pins an older revision (`1cc7b170`) that still carries the retired `mend.proof` harness, the model-review tooling and `rinse`'s canon mode; its migration to `mend.pipeline` is APPS-566 (brief: `~/ai/campaigns/canon-mymend-2026-10-18/handoffs/EPST.md`). Do not move that pin to a revision from 0.2.0 on until the migration lands: its proof-drive lane and its read-through imports would fail.

</blockquote>

## Development

<blockquote class="artificial-prose">

GitLab is the canonical development forge. Any GitHub repository is a read-only source mirror; report issues and propose changes at <https://gitlab.com/doering-ai/libs/mend>.

</blockquote>

## Versioning

<blockquote class="artificial-prose">

This project follows [semantic versioning](https://semver.org). A `1.0.0` release requires the maintainer's stability promise in `RELEASE.md`.

</blockquote>
