Metadata-Version: 2.5
Name: folderfuse
Version: 0.1.0
Summary: High-performance codebase context packager, AST skeletonizer, and multi-model token optimizer for LLMs.
Project-URL: Homepage, https://github.com/Samm-G/folder-fuse
Project-URL: Repository, https://github.com/Samm-G/folder-fuse
Project-URL: Issues, https://github.com/Samm-G/folder-fuse/issues
Author: Latent Oxygen Studios
License: MIT
License-File: LICENSE
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Code Generators
Classifier: Topic :: Utilities
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: fastapi>=0.110.0
Requires-Dist: nbformat>=5.9.0
Requires-Dist: openpyxl>=3.1.2
Requires-Dist: pathspec>=0.12.1
Requires-Dist: pypdf>=4.0.0
Requires-Dist: pyperclip>=1.8.2
Requires-Dist: python-docx>=1.1.0
Requires-Dist: pyyaml>=6.0.1
Requires-Dist: rich>=13.7.0
Requires-Dist: tiktoken>=0.7.0
Requires-Dist: tokenizers>=0.19.0
Requires-Dist: tree-sitter-c-sharp>=0.21.0
Requires-Dist: tree-sitter-c>=0.21.0
Requires-Dist: tree-sitter-cpp>=0.21.0
Requires-Dist: tree-sitter-go>=0.21.0
Requires-Dist: tree-sitter-java>=0.21.0
Requires-Dist: tree-sitter-javascript>=0.21.0
Requires-Dist: tree-sitter-python>=0.21.0
Requires-Dist: tree-sitter-rust>=0.21.0
Requires-Dist: tree-sitter-typescript>=0.21.0
Requires-Dist: tree-sitter>=0.22.0
Requires-Dist: typer>=0.12.0
Requires-Dist: uvicorn[standard]>=0.28.0
Provides-Extra: dev
Requires-Dist: httpx>=0.27.0; extra == 'dev'
Requires-Dist: mypy>=1.9.0; extra == 'dev'
Requires-Dist: pre-commit>=3.7.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23.0; extra == 'dev'
Requires-Dist: pytest-cov>=5.0.0; extra == 'dev'
Requires-Dist: pytest>=8.0.0; extra == 'dev'
Requires-Dist: ruff>=0.4.0; extra == 'dev'
Requires-Dist: types-openpyxl>=3.1.5.20240307; extra == 'dev'
Requires-Dist: types-pyyaml>=6.0.12.20240311; extra == 'dev'
Description-Content-Type: text/markdown

# FolderFuse

[![PyPI Version](https://img.shields.io/pypi/v/folderfuse.svg)](https://pypi.org/project/folderfuse/)
[![Python Version](https://img.shields.io/badge/python-3.10%2B-blue.svg)](https://www.python.org/downloads/)
[![CI Status](https://github.com/Samm-G/folder-fuse/actions/workflows/ci.yml/badge.svg)](https://github.com/Samm-G/folder-fuse/actions)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Air-Gapped](https://img.shields.io/badge/telemetry-zero--egress-success.svg)](#security--privacy)

High-performance codebase context packager, AST skeletonizer, and multi-model token optimizer engineered for Large Language Models (LLMs).

---

## Overview

**FolderFuse** packages local codebases, documentation repositories, and mixed-format folders into model-ready, token-optimized context bundles. It operates under a strict **dual-entry architecture**: a headless Unix-pipe CLI for terminal power users and automated CI/CD pipelines, alongside an embedded local Web UI for interactive preparation.

FolderFuse is strictly an **exporter**—it never dispatches requests directly to external LLM APIs, ensuring workflows remain 100% private, zero-cost, and completely air-gapped.

```
┌──────────────────────────────────────────────────────────────────┐
│                         FolderFuse Core                          │
│                                                                  │
│  ┌───────────────────────┐             ┌──────────────────────┐  │
│  │   Headless CLI        │             │   FastAPI Engine     │  │
│  │  (Typer / Rich)       │             │  (Local 127.0.0.1)   │  │
│  └──────────┬────────────┘             └──────────┬───────────┘  │
│             │                                     │              │
│             │                         ┌───────────┴───────────┐  │
│             │                         │  Embedded React UI    │  │
│             │                         │  (Vite + Tailwind)    │  │
│             │                         └───────────┬───────────┘  │
│             └───────────────────┬─────────────────┘              │
│                                 ▼                                │
│  ┌─────────────────────────────────────────────────────────────┐ │
│  │                     Processing Pipeline                     │ │
│  │                                                             │ │
│  │  1. Filesystem Crawler (.gitignore, root jail containment)  │ │
│  │  2. Document Extractors (.pdf, .docx, .xlsx, .ipynb)        │ │
│  │  3. AST Skeletonizer (Tree-Sitter multi-language parser)    │ │
│  │  4. Security Sanitizer (Zero-leak session deduplication)    │ │
│  │  5. Multi-Model Tokenizers (Claude, Gemini 3, DeepSeek V4) │ │
│  │  6. KV-Cache Optimizer (5-step layout ordering engine)      │ │
│  └──────────────────────────────┬──────────────────────────────┘ │
│                                 ▼                                │
│                   Export: Stdout / File / Clipboard              │
└──────────────────────────────────────────────────────────────────┘
```

---

## Core Capabilities

- **Dual-Entry Architecture:**
  - **Headless POSIX CLI:** Optimized for Unix piping (`folderfuse pack . | pbcopy`). Payload directed strictly to `stdout`; progress spinners, logs, and token meters routed to `stderr`.
  - **Embedded Web Dashboard:** Single-command boot (`folderfuse ui .`) launching an air-gapped React/Vite dashboard served directly from Python wheel assets on `127.0.0.1`.
- **Multi-Language AST Skeletonization (Tree-Sitter):**
  Prunes function/method implementation bodies while preserving class definitions, method signatures, return types, docstrings, and top-level constants/type aliases. Supports Python, TypeScript, JavaScript, Go, Rust, Java, C, C++, and C#.
- **Prompt-Cache Optimization (Anthropic & Gemini):**
  Structures prompt layout deterministically into a 5-step hierarchy to maximize KV-cache hit rates:
  1. System instructions prefix
  2. ASCII repository directory tree
  3. Reference documentation (`*.md`, `docs/**`)
  4. Stable source code
  5. Ephemeral / working target files placed at the prompt tail (Git-aware + glob overrides)
- **Multi-Model Tokenizer Engine:**
  100% offline exact token calculations for **Anthropic Claude** (Claude 3.5 Sonnet, Haiku), **Google Gemini** (Gemini 3.8 Flash, 3.7 Flash, 3.5 Flash), **DeepSeek** (DeepSeek-V4.1-Flash, DeepSeek-V4-Pro-0813), and **OpenAI** (GPT-4o, o200k/cl100k BPE).
- **Zero-Leak Secret Sanitization:**
  Detects credentials (OpenAI, Anthropic, AWS, GitHub PATs, JWTs, private keys) and replaces them with deterministic synthetic tokens (`[REDACTED:OPENAI_KEY:1]`), preserving cross-file references without leaking raw secrets.
- **Rich Document Parsing & Statistical Data Profiling:**
  - **Spreadsheets (`.xlsx`, `.csv`):** Markdown tables with configurable row/column caps and an automated statistical data profile summary (null percentages, unique counts, numeric stats).
  - **Jupyter Notebooks (`.ipynb`):** Strips heavy base64 image plots while preserving code cells and execution output streams.
  - **Word Docs (`.docx`) & PDFs (`.pdf`):** Extracts headings, bullet lists, and page-by-page text.

---

## Installation

Install `folderfuse` from PyPI:

```bash
pip install folderfuse
```

Or using `uv`:

```bash
uv add folderfuse
```

Or run directly without permanent installation via `pipx`:

```bash
pipx run folderfuse ui
```

---

## Quickstart

### 1. Headless CLI (`folderfuse pack`)

Package the current repository into an Anthropic Claude XML prompt and copy it directly to your system clipboard:

```bash
folderfuse pack . --model claude --clipboard
```

Stream directly to standard output for Unix pipelines:

```bash
# Pipe directly into macOS clipboard
folderfuse pack ./my-project -q | pbcopy

# Save formatted markdown context bundle to disk
folderfuse pack ./my-project --format markdown --output bundle.md
```

Apply AST skeletonization to secondary libraries:

```bash
folderfuse pack . --skeleton "src/utils/**,src/legacy/**" --strict
```

### 2. Interactive Local Web Dashboard (`folderfuse ui`)

Launch the local web server and open the dashboard in your default browser:

```bash
folderfuse ui /path/to/project
```

Options for UI server:

```bash
folderfuse ui . --port 8080 --no-browser
```

---

## CLI Flag Reference (`folderfuse pack`)

| Flag                     | Short | Default             | Description                                                            |
| :----------------------- | :---- | :------------------ | :--------------------------------------------------------------------- |
| `PATH`                   |       | `.`                 | Target directory path to inspect and package.                          |
| `--model`                | `-m`  | `claude-3-5-sonnet` | Tokenizer profile (`claude`, `gemini`, `openai`, `deepseek`).          |
| `--format`               | `-f`  | `xml`               | Output layout (`xml`, `markdown`, `json`, `raw`).                      |
| `--output`               | `-o`  | `None`              | Destination file path to write bundle to.                              |
| `--clipboard`            | `-c`  | `False`             | Copy generated context bundle to system clipboard.                     |
| `--skeleton`             |       | `None`              | Glob patterns for AST skeletonization (repeatable or comma-separated). |
| `--exclude`              |       | `None`              | Additional glob patterns to exclude from ingestion.                    |
| `--working`              |       | `None`              | Globs for volatile files to place at prompt tail.                      |
| `--no-gitignore`         |       | `False`             | Disable automatic `.gitignore` rule evaluation.                        |
| `--follow-symlinks`      |       | `False`             | Permit traversing symlinks pointing outside root jail.                 |
| `--scrub-secrets`        |       | `True`              | Toggle automated secret and credential masking.                        |
| `--omit-sensitive-files` |       | `False`             | Omit sensitive files (`.env`, `id_rsa`) instead of scrubbing.          |
| `--max-size-kb`          |       | `500`               | Skip text files exceeding threshold in KB.                             |
| `--instructions`         |       | `None`              | Custom system framing instructions for prompt header.                  |
| `--strict`               |       | `False`             | Exit with code `3` if total tokens exceed model context limit.         |
| `--json-meta`            |       | `False`             | Emit structured JSON diagnostic metadata to stderr.                    |
| `--quiet`                | `-q`  | `False`             | Suppress all diagnostic banners, tables, and spinners.                 |

### POSIX Exit Codes
- `0`: Success (Context bundle generated cleanly).
- `1`: General runtime or filesystem write error.
- `2`: Target directory not found or invalid path.
- `3`: Context window budget overflow (when `--strict` is set).

---

## Configuration File (`folderfuse.yaml`)

FolderFuse supports repository-level configuration files (`folderfuse.yaml` or `.folderfuse.json`) located at the target directory root:

```yaml
version: "1"
settings:
  default_model: "claude-3-5-sonnet"
  default_format: "xml"
  max_file_size_kb: 500
  scrub_secrets: true
  strict: false
  include_instructions: true
  instruction_text: "You are an expert systems engineer evaluating this repository."

exclusions:
  - "dist/**"
  - "build/**"
  - "**/*.spec.ts"

skeletonize:
  - "legacy/**/*.py"
  - "src/vendor/**/*.ts"

working_files:
  - "src/active_feature.py"

custom_redactions:
  - name: "INTERNAL_CODENAME"
    regex: "(?i)project-zeus"
    replace: "[PROJECT_ZEUS]"
```

*Precedence:* Built-in Defaults < User Global Config (`~/.config/folderfuse/config.yaml`) < Target Repo Config (`folderfuse.yaml`) < CLI Flags.

---

## Security & Privacy

- **Air-Gapped Operation:** FolderFuse makes zero network requests, contains zero analytics SDKs, and bundles all frontend and grammar assets locally inside the Python wheel.
- **Localhost-Only Binding:** The embedded FastAPI server binds strictly to loopback interfaces (`127.0.0.1`).
- **Root Jail Containment:** Symbolic links resolving outside the designated root directory are blocked by default to prevent path traversal attacks.

---

## License

Distributed under the terms of the [MIT License](LICENSE).  
Copyright © 2026 Latent Oxygen Studios.