Metadata-Version: 2.4
Name: git-reaper
Version: 0.8.0
Summary: A spooky utility for data mining git repositories (and any folder foolish enough to hold still).
Project-URL: Homepage, https://github.com/jmcmeen/git-reaper
Author: John McMeen
License: MIT License
        
        Copyright (c) 2026 John McMeen
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE
Keywords: analysis,cli,git,llm,repository
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Software Development :: Version Control :: Git
Requires-Python: >=3.10
Requires-Dist: pathspec>=0.12
Requires-Dist: rich>=13.0
Requires-Dist: tomli>=2.0; python_version < '3.11'
Requires-Dist: typer>=0.12
Provides-Extra: all
Requires-Dist: gitpython>=3.1; extra == 'all'
Requires-Dist: mcp>=1.10; extra == 'all'
Requires-Dist: textual>=0.86; extra == 'all'
Requires-Dist: tiktoken>=0.7; extra == 'all'
Requires-Dist: tree-sitter-language-pack>=0.13; extra == 'all'
Requires-Dist: tree-sitter>=0.23; extra == 'all'
Provides-Extra: bones
Requires-Dist: tree-sitter-language-pack>=0.13; extra == 'bones'
Requires-Dist: tree-sitter>=0.23; extra == 'bones'
Provides-Extra: git
Requires-Dist: gitpython>=3.1; extra == 'git'
Provides-Extra: mcp
Requires-Dist: mcp>=1.10; extra == 'mcp'
Provides-Extra: tokens
Requires-Dist: tiktoken>=0.7; extra == 'tokens'
Provides-Extra: tui
Requires-Dist: textual>=0.86; extra == 'tui'
Description-Content-Type: text/markdown

# git-reaper

> A spooky utility for data mining git repositories (and any folder foolish enough to hold still).

`git-reaper` reaps structured knowledge from repositories: it clones, gathers,
concatenates, and packs the contents of a git repo (or a plain directory) into
clean, portable artifacts. Library first; the CLI is a thin adapter over
`git_reaper.core`.

## Install

```sh
uv tool install git-reaper   # or: pip install git-reaper
```

Both `reaper` and `git-reaper` land on your PATH (the long form is the
fallback if the REAPER DAW already owns the short one).

## Commands

### Reaping and packing

| Command | What it does |
| --- | --- |
| `harvest` | Gather files matching a pattern (default `*.md`) from a path or repo URL and concatenate them into one artifact with a provenance header and per-file dividers. |
| `limbs` | Hierarchical file listing (the tree, limb by limb) as markdown or JSON. Depth limits, dirs-only, sizes, line counts, ignore rules. |
| `conjure` | Bundle a repo into a single LLM-ingestible file: tree first, then every text file inlined with spec'd delimiters. `--sha256` for verifiable hashes, `--split-tokens N` to shard into context-window-sized parts. |
| `reanimate` | The inverse of `conjure`: reconstruct a directory tree from a packed artifact. `--verify` checks per-file hashes; path traversal is refused outright. |
| `census` | File-type census: counts, sizes, line counts, language breakdown, token estimate. Size a repo before packing it. |
| `distill` | Skill harvesting: read a repo and emit a portable Agent Skill (`SKILL.md` + `reference/`) that teaches a model to work there — conventions, real build/test/lint commands, the structural map, the files that break most, and who to ask (`--anon` for roles). Deterministic, no network, no model calls; `--polish CMD` optionally pipes each draft's prose through your own agent command (stamps and frontmatter protected). `--profile {repo,stack,onboarding}` sets the voice; `--check skills/<name>/` exits 3 when the code has moved past the stamped sha. `necropolis distill` harvests a whole fleet into a skill library with a routing index skill at its root. |
| `unfinished` | Scan for TODO / FIXME / HACK / XXX markers, with authors via git blame and `--age` for how long each has haunted. |
| `grimoire` | Show effective configuration, where each value came from, and stored recipes. |
| `cast` | Run a saved recipe from the grimoire instead of retyping nine flags. |
| `pulse` | Signs-of-life check: git present, optional extras, cache health. |
| `banish` | Clear the catacombs (the clone cache). `--older-than 7d` for partial exorcisms. |
| `summon` | Launch the Sanctum, the interactive Textual TUI (needs the `[tui]` extra): a Dracula-themed workbench of chambers reached from a home crypt map. The Altar runs any analysis ritual (options, preview, save, cursed badge); the Grimoire composes recipes visually and inscribes them in `.reaperrc` for `cast`; the Incantation console is an assisted CLI with `/` commands, fuzzy menus, live flag validation, and history; the Necropolis board reaps a whole `necropolis.toml` fleet with per-grave fates; the Reliquary triages `exhume`/`omens`/`plague`/`rot` on one severity-sorted slab; the Séance table pairs the souls heatmap with an hour-by-hour commit explorer and a two-ref scry. Number keys jump chambers, escape returns home, Ctrl+P's palette knows every door and theme. Nothing is TUI-trapped: recipes cast headless and every console line is a real `reaper` invocation. |
| `commune` | Serve the read-only rituals to agents as an MCP server (needs the `[mcp]` extra): every analysis ritual becomes an agent-callable tool returning the same provenance-stamped JSON, over stdio (default) or `--http HOST:PORT`. Rooted to the launch source unless `--root`/`--host` widen the circle; the writing rituals (`resurrect`, `reanimate`, `banish`) appear only with `--allow-write`, `veil` scrubs text in flight, and `plague` stays offline without `--allow-network`. Publishes the grimoire, tombstone, and census as MCP resources plus ready-made audit/pack/explain prompts. |

### Git necromancy (history mining)

| Command | What it does |
| --- | --- |
| `chronicle` | Commit history (SHA, author, date, message, churn) to markdown, JSON, or CSV. `--changelog` groups commits by tag. |
| `souls` | Contributor stats: commits, lines added/removed, first/last seen, bus factor. `--heatmap` draws a day-of-week x hour activity grid and flags the repo's witching hour. |
| `haunt` | Code churn and hotspots: files ranked by change frequency and churn, the classic bug-risk proxy. |
| `autopsy <path>` | Deep single-file exam: creation commit, rename history (`--follow`), authors over time, churn, and a blame-based line-age summary. |
| `graveyard` | Every file that ever lived and died: path, date of death, the fatal commit, and its author. |
| `resurrect <path>` | Restore a dead file's last living bytes into the working tree or `--out`. Path traversal is refused outright. |
| `ghosts` | Branch hygiene: branches ranked by abandonment, with merged, gone-upstream, and (past `--than 90d`) stale flags. |
| `rot` | Staleness report: surviving files ranked by how long they have gone untouched. |
| `tombstone` | A stats card for demos and READMEs (born, age, commits, souls, last words, witching hour) as ASCII tombstone art, or JSON. |

History commands need real history, so remote sources are cloned full-depth
(a previously shallow catacombs clone is unshallowed automatically).

### Dark arts (security, risk, and forensics)

| Command | What it does |
| --- | --- |
| `exhume` | Scan the full history for committed secrets (API keys, tokens, private keys) via regex signatures plus entropy. Reports commit, path, rule, and a masked preview, never the full secret. `--baseline` suppresses known findings; `--fail-on {any,high}` gates CI. |
| `veil` | Scrub secrets and configured patterns from any artifact (or stdin) before it leaves the crypt, replacing each match with `[VEILED:rule-name]`. Shares one rules engine with `exhume`; also inline as `conjure --veil`. |
| `omens` | Composite per-file risk prophecy: a weighted blend of churn, bug-fix density, recency, and size. `--lens {churn,bugs,age,all}`, weights configurable in the grimoire. Hints, not fate. |
| `doppelgangers` | Find duplicate files by content hash. Reports clusters and reclaimable space. |
| `bloat` | Largest files in the working tree and, for repos, the blobs deleted from the tree but still weighing down `.git`. |

### Deeper necromancy

| Command | What it does |
| --- | --- |
| `bones` | Strip implementation, keep structure: every file's imports, signatures, and docstrings. Python via `ast`; other languages via the `git-reaper[bones]` (tree-sitter) extra. |
| `scry` | Compare two refs: churn, most-changed files, contributors, and souls first seen in the range. |
| `plague` | Opt-in and network-using: read dependency manifests and check pinned versions against the OSV database. `--offline` parses manifests only. The only command that leaves the crypt. |
| `necropolis` | Fan any source-taking command across every grave in a `necropolis.toml` manifest (or a GitHub `--org`). Per-repo artifacts plus a combined `INDEX.md`. |

Analysis commands add `--format html` for a self-contained, dark-themed
report. `exhume --fail-on`, `omens --fail-over`, `plague --fail-on`, and a
cursed grave in `necropolis` all exit `3` for one-line CI gates.

```sh
reaper harvest https://github.com/Textualize/rich --pattern "*.md" -o RICH.md
reaper conjure . --sha256 --split-tokens 100000 -o PACKED.md
reaper reanimate PACKED.md --out risen/ --verify
reaper census . --format csv | head
reaper distill . --out skills/git-reaper/     # harvest an Agent Skill
reaper distill --check skills/git-reaper/     # is the skill still true? (exit 3 if stale)
reaper distill . --polish 'claude -p "tighten this skill draft"'  # your model, your key
reaper necropolis distill --org acme --out-dir skills/  # a skill library + routing skill
reaper unfinished . --age
reaper cast nightly-pack
reaper limbs . --format json | jq .file_count
reaper banish --older-than 7d

reaper chronicle . --changelog
reaper souls . --heatmap
reaper haunt . -n 20 --format json | jq '.hotspots[0]'
reaper autopsy src/git_reaper/cli.py
reaper graveyard . && reaper resurrect old/module.py --out risen/
reaper ghosts . --than 90d
reaper tombstone .

reaper exhume . --fail-on any                 # CI secret gate (exit 3)
reaper conjure . --veil -o SAFE.md            # pack, scrubbing secrets
reaper omens . --lens churn -n 20             # riskiest files first
reaper doppelgangers . && reaper bloat .      # duplicates, then heft
reaper bones . -o MAP.md                      # structure without the flesh
reaper scry v1.0.0 HEAD -o DELTA.md           # what changed between releases
reaper plague . --offline                     # dependency advisories (opt-in net)
reaper necropolis harvest --tag docs --out-dir out/   # fan out over a manifest
reaper haunt . --format html -o hotspots.html # self-contained dark report

reaper summon .            # interactive TUI (pip install "git-reaper[tui]")
reaper commune .           # MCP server over stdio (pip install "git-reaper[mcp]")
reaper commune . --http 127.0.0.1:6666 --root ~/repos   # a shared reaper for a team
```

Recipes live in `.reaperrc` (or `[tool.reaper]` in pyproject.toml), alongside
custom secret rules and tunable omen weights:

```toml
[recipes.nightly-pack]
command = "conjure"
args = [".", "--sha256", "--split-tokens", "100000", "--out", "PACKED.md"]
description = "the whole crypt, sharded for the model"

# extend the exhume/veil engine with your own signatures
[rules.internal-host]
pattern = "[a-z0-9-]+\\.corp\\.example\\.com"
severity = "medium"
veil_only = true   # veil redacts it; exhume does not report it as a secret

# bless a team's `commune` MCP server with shared defaults
[commune]
roots = ["~/repos"]
allow_network = true

# tune the omens blend (defaults shown)
[omens]
churn = 0.35
bugs = 0.30
age = 0.20
size = 0.15
```

## Optional extras

The base install is lean. Heavier machinery lives behind extras:

```sh
pip install "git-reaper[bones]"    # tree-sitter: bones for JS/TS/Go/Rust/Java/...
pip install "git-reaper[tui]"      # textual: the Dracula `reaper summon` TUI
pip install "git-reaper[tokens]"   # tiktoken: exact token counts
pip install "git-reaper[git]"      # GitPython backend (GIT_REAPER_BACKEND=gitpython)
pip install "git-reaper[mcp]"      # the `reaper commune` MCP server
pip install "git-reaper[all]"      # everything
```

Third-party "rituals" extend the CLI through the `git_reaper.rituals` entry
point: a package that registers a Typer sub-app appears as `reaper <name>`.

## Behavior you can rely on

- **Artifacts to stdout (or `--out`); narration to stderr.** Piping is always safe.
- **Deterministic output.** Same repo state + same flags = byte-identical
  artifact (only the provenance timestamp moves).
- **Provenance by default.** Every artifact opens with source, ref/sha,
  timestamp, tool version, and the exact invocation.
- **Ignore rules honored.** `.gitignore` + `.reaperignore` + `--exclude`
  globs; `.git` is never reaped.
- **Caps with receipts.** Size caps and binary detection skip files loudly,
  never silently.
- **The catacombs.** Remote clones cache under
  `~/.cache/git-reaper/catacombs/<host>/<owner>/<repo>`, shallow, reused
  across runs. Local `file://` sources are buried flat
  (`localhost/<name>-<digest>`) to stay inside Windows path limits.
- **Published schemas.** Every JSON-emitting command prints its JSON schema
  with `--schema`.
- **`--plain` / `NO_COLOR`** produce clean ASCII; non-tty output auto-disables
  the theatrics.
- **No telemetry.** The dead tell no tales.

## Library use

```python
from git_reaper.core.source import resolve_source
from git_reaper.core.harvest import harvest
from git_reaper.formatters.markdown import write_harvest

repo = resolve_source("https://github.com/Textualize/rich").repo
result = harvest(repo, patterns=("*.md",))
with open("RICH.md", "w") as fh:
    write_harvest(result, fh)
```

## Development

Everything runs through `uv` and the Makefile:

```sh
make setup      # create venv, install deps
make check      # lint + typecheck + tests (the full gauntlet)
make fmt        # auto-format and fix lint findings
make test       # pytest
make cov        # pytest with coverage
make docs       # serve the docs locally
make run ARGS="tree ."
make build      # sdist + wheel
```

## License

MIT. Rest in peace.
