Metadata-Version: 2.5
Name: repoharvester
Version: 0.3.2
Summary: Clone a git repo or read a folder and compile its text files into one file for LLM context. CLI plus optional GUI, no dependencies.
Project-URL: Homepage, https://github.com/octrow/RepoHarvester
Project-URL: Changelog, https://github.com/octrow/RepoHarvester/blob/main/CHANGELOG.md
Project-URL: Issues, https://github.com/octrow/RepoHarvester/issues
License-Expression: MIT
License-File: LICENSE
Keywords: chatgpt,claude,cli,code-to-text,git,llm,llm-context,prompt,repository
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development
Classifier: Topic :: Utilities
Requires-Python: >=3.12
Provides-Extra: gui
Requires-Dist: customtkinter>=5.2; extra == 'gui'
Description-Content-Type: text/markdown

<p align="center">
  <img src="docs/hero.svg" width="100%" alt="RepoHarvester: any git repository as one text file, with a token estimate for each file. Shown: the start of the real output for psf/requests, 92 files and about 154.6k tokens.">
</p>

[![CI](https://github.com/octrow/RepoHarvester/actions/workflows/ci.yml/badge.svg)](https://github.com/octrow/RepoHarvester/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/repoharvester)](https://pypi.org/project/repoharvester/)
[![Python](https://img.shields.io/pypi/pyversions/repoharvester)](https://pypi.org/project/repoharvester/)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue)](LICENSE)

**English** | [Русский](README.ru.md)

Point it at a git URL or a local folder. You get all text files in one file, ready to paste into ChatGPT or Claude.
A command-line tool with an optional GUI. No runtime dependencies.

```shell
uvx repoharvester https://github.com/psf/requests
```
```
Cloning https://github.com/psf/requests
Wrote 92 files to output/requests_all_files.txt, ~154.6k tokens (0 non-UTF-8 skipped)
```

## What you get
`output/requests_all_files.txt` starts with a list of every file and its size in tokens, then the files themselves:
```
## requests
### Files: 92, ~154.6k tokens
.coveragerc  ~16
HISTORY.md  ~16.1k
src/requests/models.py  ~10.4k
...

### src/requests/api.py
"""
requests.api
...
### end of file
```
The list shows the model the shape of the repository and shows you which files eat the context. Token counts are estimates (~4 characters per token). Files that are not UTF-8 are listed in `skipped_files.txt` next to the output.

## Features
- Works with SSH/HTTPS git URLs (shallow clone into a temp dir) or local folders.
- Skips binary/noise by default: media, archives, executables, lock files, `node_modules`, virtualenvs, caches, `.git`, `.github`, and more.
- Optional comment removal for common languages. String literals are kept, so `"http://..."` and `"#fff"` stay intact. Python uses the real tokenizer; other languages use regexes.
- Respects `.gitignore`: in a git repo only tracked and untracked-but-not-ignored files are harvested.
- Glob filters (`--include '*.py'`, `--exclude-glob 'tests/*'`), a per-file size limit, and `-o -` to print straight to stdout.

## Install
Requires Python 3.12+ and git.

```shell
uv tool install repoharvester            # or: pipx install repoharvester
uv tool install "repoharvester[gui]"     # with the GUI
```

## Usage
```shell
repoharvester <repo-url-or-folder> [options]
```

| Option | Description |
|---|---|
| `-r`, `--remove` | Remove comments from code files |
| `--no-skip GROUP[,GROUP]` | Include groups that are skipped by default (repeatable) |
| `--max-size KB` | Skip files larger than this (default 1000). Files over 500 KB are logged |
| `--exclude DIR[,DIR]` | Folder names to exclude, along with their contents (repeatable) |
| `--include GLOB[,GLOB]` | Only harvest files matching these globs, e.g. `*.py,*.md` (repeatable) |
| `--exclude-glob GLOB[,GLOB]` | Skip files matching these globs, e.g. `tests/*` (repeatable) |
| `-o`, `--output-dir` | Output directory (default `output`), or `-` to print to stdout |
| `--log FILE` | Also write the log to a file |

File groups and glob rules: see [docs/filters.md](docs/filters.md).

Example:
```shell
repoharvester git@github.com:username/repo.git -r --no-skip configuration --max-size 250 --exclude vendor,docs

# only Python and Markdown, no tests, straight to the clipboard
repoharvester . --include '*.py,*.md' --exclude-glob 'tests/*' -o - | xclip -selection clipboard   # macOS: | pbcopy
```

## GUI
```shell
repoharvester-gui                         # after installing with [gui]
uv run --extra gui repoharvester-gui      # from a clone
```

<img src="docs/gui.png" width="520" alt="RepoHarvester window: repository URL and output folder fields, max file size, skip folders, only-files and skip-files globs, a strip-comments checkbox, a grid of file groups to include, and a Harvest button above an empty log">

Paste a git URL or choose a folder, tick anything extra to include, press **Harvest** (or Enter), then **Open file**.
While a clone runs the button turns into **Cancel**. Settings are remembered in `~/.config/repoharvester/gui.json`.
On Linux the project prefers the system Python: uv's bundled builds ship Tk without font antialiasing.

## Alternatives
- [repomix](https://github.com/yamadashy/repomix) (Node.js): more output formats, exact token counts, many more options. Pick it if you have Node and want those.
- [gitingest](https://github.com/coderamp-labs/gitingest) (Python): also has a website where you swap `hub` for `ingest` in a GitHub URL.
- [files-to-prompt](https://github.com/simonw/files-to-prompt) (Python): local files and folders, no cloning.

RepoHarvester is one standard-library module. It needs only Python and git.

## Development
See [CONTRIBUTING.md](CONTRIBUTING.md).

```shell
uv run --group dev pytest
uvx ruff check .
```

## License
MIT. See `LICENSE`.
