Metadata-Version: 2.4
Name: packdiff
Version: 0.3.0
Summary: Pack a repository into a single, LLM-friendly Markdown file.
Author-email: muguang1224 <137096726+muguang1224@users.noreply.github.com>
License: MIT
Project-URL: Homepage, https://github.com/muguang1224/packdiff
Project-URL: Repository, https://github.com/muguang1224/packdiff
Project-URL: Issues, https://github.com/muguang1224/packdiff/issues
Project-URL: Changelog, https://github.com/muguang1224/packdiff/blob/main/CHANGELOG.md
Keywords: llm,context,markdown,cli,prompt-engineering,codebase,ai-agents
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Documentation
Classifier: Topic :: Utilities
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: ruff>=0.6.0; extra == "dev"
Dynamic: license-file

# packdiff

**Pack a repository into a single, LLM-friendly Markdown file.**

[![CI](https://github.com/muguang1224/packdiff/actions/workflows/ci.yml/badge.svg)](https://github.com/muguang1224/packdiff/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/packdiff)](https://pypi.org/project/packdiff/)
[![Python](https://img.shields.io/badge/python-3.9%2B-blue)](https://www.python.org/downloads/)
[![Dependencies](https://img.shields.io/badge/dependencies-none-brightgreen)](#design-notes)
[![License: MIT](https://img.shields.io/badge/license-MIT-yellow.svg)](LICENSE)

[English](README.md) · [简体中文](README.zh-CN.md)

You are about to paste a codebase into an LLM. Which files? How big is the
prompt? Did you just include `.env`?

`packdiff` answers all three. It walks a repository, skips the noise and the
credentials, and writes one clean Markdown document — directory tree, language
overview, and every text file in a fenced code block — with a real token
estimate in the header.

```
packdiff . -o context.md                # pack the current repo
packdiff --max-tokens 32k -o ctx.md     # ...and fit it in a 32k window
packdiff --diff -o review.md            # ...or just what you have not committed
packdiff src/ docs/ --include '*.py'    # ...or just two trees, Python only
```

## Install

```bash
pip install packdiff
```

Or with [pipx](https://pipx.pypa.io/) if you would rather not put it inside a
virtualenv:

```bash
pipx install packdiff
```

Python 3.9 or newer. No dependencies — standard library only.

## Quick start

```bash
cd ~/projects/my-service
packdiff
```

````markdown
# packdiff · repository context pack

> Generated by packdiff v0.3.0 on 2026-10-13 10:24 UTC.
> Roots: `my-service`
> 5 files · 103 B · ~215 tokens (estimated)

## Overview

| Language | Files | Lines |
| --- | ---: | ---: |
| markdown | 1 | 3 |
| python | 2 | 3 |
| gitignore | 1 | 2 |
| text | 1 | 1 |

## Directory tree

```text
my-service/
├── src/
│   ├── __init__.py
│   └── core.py
├── .gitignore
├── README.md
└── requirements.txt
```

## Files

### `.gitignore`
```gitignore
.venv/
*.pyc
```

### `README.md`
```markdown
# Demo

A tiny service.
```

### `requirements.txt`
```text
requests>=2.31
```

### `src/__init__.py`
```python
from .core import main
```

### `src/core.py`
```python
def main():
    print("hi")
```

## Skipped as secret-like

Review these before sharing the pack; use `--include-secrets` to override.

- `.env`
````

That is the whole point: a document you can paste, diff, cache, and re-generate
when the code changes.

## What ends up in the pack

Included: every text file, including dotfiles such as `.github/workflows/ci.yml`
and `.gitignore`.

Excluded by default:

| Category | Examples | How to override |
| --- | --- | --- |
| Ignored paths | anything matched by `.gitignore`, nested ones included, plus `.git/info/exclude` | `--no-gitignore` |
| Build noise | `node_modules/`, `dist/`, `build/`, `target/`, `__pycache__/`, `.venv/`, `vendor/`, … | `--no-default-ignores` |
| Lockfiles | `package-lock.json`, `poetry.lock`, `Cargo.lock`, `go.sum`, … | `--no-default-ignores` |
| Binary files | images, archives, executables, fonts, PDFs, office documents, databases | — |
| Binary content | any file containing a NUL byte, or that fails to decode | — |
| Large files | text above `--max-file-size` (default 1 MiB) | `--max-file-size 8m` |
| Empty files | zero-byte leftovers | — |
| Secret-like files | `.env`, `*.pem`, `*.key`, `id_rsa`, `.netrc`, `credentials.json`, … | `--include-secrets` |

Every exclusion is reported on stderr, so nothing disappears silently:

```
$ packdiff
packdiff: 5 files, 103 B from my-service
packdiff: skipped .env: looks like a secret (pass --include-secrets)
```

## Reviewing only what changed

The most common way to use this tool is not "here is my repository" but "here is
what I just changed". `--diff` packs only the files that differ from a revision,
and it includes uncommitted and untracked files, because a new file is part of
the change:

```bash
packdiff --diff                      # what have I not committed yet?
packdiff --diff main                 # what does this branch add on top of main?
packdiff --diff HEAD~3 -o review.md  # the last three commits
packdiff src/ --diff                 # ...restricted to one subtree
```

It composes with everything else, so the change set still passes through
`.gitignore`, the binary check and the secret filter:

```bash
packdiff --diff --max-tokens 16k | pbcopy
```

Deletion is handled the way you would hope: a deleted file appears in the change
set from git, then fails to match anything on disk, so it simply is not packed.
Nothing changed, or everything in the change set was filtered out, is a clear
error rather than an empty pack:

```
$ packdiff --diff
packdiff: error: no files changed against 'HEAD'
```

Change detection shells out to `git diff`, so the answer is the same one
`git status` just gave you — and if git is missing, or the revision is wrong, or
you point `--diff` at a directory outside the repository, you get an error that
says so instead of a silently empty pack.

## A worked example

[`examples/demo-project.pack.md`](examples/demo-project.pack.md) is a committed
pack of a small demo project, so you can read the real output without installing
anything. The demo directory is arranged to show the filters doing their job:
`cache/` and `debug.log` are gone via `.gitignore`, `assets/logo.png` is gone as
binary, and `.env.example` survives because a template is not a secret.

```bash
python3 examples/regenerate.py    # rebuild it after changing the format
```

`tests/test_examples.py` regenerates the pack and compares it byte for byte
against the committed file, so the example cannot quietly drift out of date.

## Token budgets

`--max-tokens` keeps the pack inside a context window instead of blowing past
it mid-paste.

```bash
packdiff --max-tokens 32k -o ctx.md
```

Files are admitted in a deliberate order: first the ones whose *name* suggests
architecture — README, `pyproject.toml` / `package.json`, entry points such as
`main.py` or `index.ts` — and then the rest largest-first, on the theory that a
400-line module tells you more about a repo than a 3-line `__init__.py`.
Whatever does not fit is listed at the bottom of the document rather than
dropped silently:

```markdown
## Deferred by token budget

These files matched but did not fit; raise `--max-tokens` to include them.

- `src/packdiff/walk.py` (11.9 KiB, ~3,049 tokens)
```

A first pass holds back 3 % of the budget (at least 120 tokens) for the header
and the tree. Per-file estimates cannot know how large those will turn out to be,
though, so the finished document is then measured, and the weakest admitted file
is deferred and the document re-rendered until the estimate actually fits.
`--max-tokens` is a guarantee on the rendered output, not an aspiration.

### How honest is the estimate?

There is no tokenizer in this repo, by design — a 200 MB wheel for a
dependency-free CLI would be absurd. Instead the estimator charges ASCII
characters at `1 / 4` tokens each (the familiar "~4 characters per token" rule)
and non-ASCII characters at one token each, which covers CJK text properly
instead of underestimating it fourfold. Expect to land within roughly ±10 % of a
BPE tokenizer on ordinary source code. Tune it with `--token-ratio` if you know
your model better than the rule of thumb does.

## Ignore rules

Three layers, evaluated in this order:

1. **Built-in denylist** — build artefacts and lockfiles (`--no-default-ignores` to disable).
2. **`.gitignore`** — including nested files, each anchored to its own directory, and `.git/info/exclude` (`--no-gitignore` to disable).
3. **`.packdiffignore`** — gitignore syntax, read from the root of the first path you pass, for exclusions you do not want to commit (`--ignore-file` to point elsewhere).

`--include` and `--exclude` accept the same glob syntax and are applied last:

```bash
packdiff --include '*.py'                     # only Python files
packdiff --include '*.py' --include '*.toml'  # repeatable, additive
packdiff --exclude 'tests/**' --exclude '*.snap'
packdiff --exclude 'src/**' --exclude '!src/public/**'   # negation re-includes
```

Supported gitignore syntax: `#` comments, `\#` / `\!` escapes, backslash
escaping anywhere in a pattern (`foo\ bar` matches `foo bar`), trailing-space
stripping, `!` negation, trailing `/` for directory-only patterns, leading `/`
for anchoring, `*`, `?`, `[a-z]` within a path segment, and `**` as a whole
segment. A `**` at the end of a pattern matches everything *inside* a directory
but not the directory itself; anywhere else it matches zero or more directories.
A pattern that matches a directory excludes everything inside it, with or
without a trailing slash. Re-including a file whose parent directory is excluded
does not work — that is git's rule too, and packdiff enforces it by pruning
excluded directories rather than descending into them.

This is the one part of the tool where "close enough" would quietly produce
wrong answers, so it ships with an oracle rather than only with unit tests. Run
it against any repository you trust and it will tell you where packdiff and git
disagree:

```
$ python3 tools/gitignore_oracle.py /path/to/repo
compared 301 paths: 0 mismatches
```

Across this repository, a synthetic corpus of edge cases, and the checked-out
trees of Flask, Requests, Express and Click — over a thousand paths in total —
it reported zero disagreements. It found three real bugs during development:
escapes inside patterns, the trailing-`**` rule, and contents of a matched
directory. All three now have tests that cite git as the authority.

## Safety

The output of this tool is, by construction, something you paste into someone
else's model. That makes the secret filter a feature, not a nicety: files such
as `.env`, `*.pem`, `*.key`, `id_rsa`, `.netrc` and `credentials.json` are
skipped by default and reported by name. Templates stay: `.env.example`,
`.env.sample`, `id_rsa.pub`.

```bash
packdiff --include-secrets      # only when you really do want them
```

The filter is a heuristic. It is not a substitute for reading what you are about
to send.

## Command reference

| Option | Default | Description |
| --- | --- | --- |
| `PATH...` | `.` | Files or directories to pack. Multiple roots are merged into one document. |
| `-o, --output FILE` | stdout | Write the pack to `FILE`. `-` means stdout. |
| `--max-tokens N` | unlimited | Keep the pack under roughly `N` tokens. Accepts `32k`, `1m`. |
| `--diff [REV]` | off | Only pack files changed against `REV` (default `HEAD`), including uncommitted and untracked files. |
| `--include GLOB` | — | Only pack matching paths. Repeatable. |
| `--exclude GLOB` | — | Skip matching paths. Repeatable. |
| `--ignore-file FILE` | `.packdiffignore` | Extra ignore file, gitignore syntax. |
| `--no-gitignore` | off | Do not read `.gitignore` / `.git/info/exclude`. |
| `--no-default-ignores` | off | Do not apply the built-in denylist. |
| `--include-secrets` | off | Do not skip secret-like files. |
| `--follow-symlinks` | off | Descend into symlinked directories (loop-protected). |
| `--max-file-size SIZE` | `1m` | Skip text files larger than `SIZE`. Accepts `512k`, `8m`. |
| `--encoding ENC` | `utf-8` | Encoding used to read files. |
| `--token-ratio R` | `4.0` | ASCII characters per token used by the estimator. |
| `--no-tree` | off | Omit the directory tree. |
| `--ascii` | off | Use ASCII tree characters (`|--`, `` `-- ``). |
| `--list` | off | List the files that would be packed, without their content. |
| `-q, --quiet` | off | Suppress the stderr summary. |
| `--version` | | Print the version and exit. |

Exit status: `0` success, `1` runtime error, `2` usage error.

## Recipes

**Into a prompt from the shell**

```bash
packdiff src/ --max-tokens 16k | pbcopy            # macOS
packdiff src/ --max-tokens 16k | clip              # Windows
```

**A code review pack** — just the change, for pasting into a reviewer:

```bash
packdiff --diff --max-tokens 60k -o review.md       # uncommitted work
packdiff --diff main --max-tokens 60k -o review.md   # a whole branch
```

**Implementation plus tests, nothing else:**

```bash
packdiff src/ tests/ --include '*.py' --max-tokens 100k -o review.md
```

**Reproducible output** for CI artefacts or committed snapshots: set
`SOURCE_DATE_EPOCH` and the timestamp in the header becomes deterministic.

```bash
SOURCE_DATE_EPOCH=0 packdiff . -o context.md
```

**See the cost before you commit to it:**

```bash
packdiff --list          # tokens and bytes per file
packdiff --list | sort -k2 -n | tail   # the heaviest files
```

## Design notes

A few decisions worth knowing about, in case you want to change them.

* **No dependencies.** Not even a TOML parser for config — `.packdiffignore` is
  gitignore syntax read by an engine that had to exist anyway.
* **Prune, don't filter.** Excluded directories are never descended into, which
  keeps the walk fast on `node_modules` and matches git's own re-inclusion
  semantics for free.
* **Fences size themselves.** A pack containing a Markdown file that talks about
  code fences would break with a fixed ```` ``` ````; the renderer emits one
  backtick longer than the longest run in the file.
* **Unicode output, ASCII fallback.** If the output stream cannot encode
  box-drawing characters (a Windows console, say), the tree silently degrades to
  `|--` / `` `-- ``.
* **`--diff` asks git rather than guessing.** Parsing the index by hand would be
  another thing to get wrong; `git diff` already knows, and the answer matches
  what the user saw in `git status` a second earlier.

## Development

```bash
git clone https://github.com/muguang1224/packdiff
cd packdiff
python -m pip install -e ".[dev]"

python -m pytest            # the full suite; no network, no fixtures on disk
ruff check . && ruff format --check .

# Optional: check the ignore engine against real git (needs git on PATH)
python3 tools/gitignore_oracle.py .

# Optional: rebuild the committed example pack
python3 examples/regenerate.py
```

The test suite builds its fixtures in temporary directories, so it runs on
Linux, macOS and Windows without any setup.

## License

MIT — see [LICENSE](LICENSE).
