Metadata-Version: 2.4
Name: tokensafe
Version: 1.0.1
Summary: Autonomous Zero-Loss Token Economy & Context Optimizer for AI Coding Agents
Author-email: TokenSafe Team <support@tokensafe.dev>
License-Expression: MIT
Project-URL: Homepage, https://tokensafe.dev
Project-URL: Repository, https://github.com/NekoSin4eG/tokensafe
Project-URL: Documentation, https://tokensafe.dev
Project-URL: Issues, https://github.com/NekoSin4eG/tokensafe/issues
Keywords: ai,llm,tokens,token-optimization,context-window,cursor,claude,antigravity,codex,mcp,mcp-server,ast,proxy
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Build Tools
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# ⚡ TokenSafe (`tokensafe`)

[![npm version](https://img.shields.io/npm/v/tokensafe.svg?color=CB3837)](https://www.npmjs.com/package/tokensafe)
[![PyPI version](https://img.shields.io/pypi/v/tokensafe.svg?color=3775A9)](https://pypi.org/project/tokensafe/)
[![License: MIT](https://img.shields.io/badge/License-MIT-emerald.svg)](https://opensource.org/licenses/MIT)
[![SWE-bench: Verified](https://img.shields.io/badge/SWE--bench-61.8%25_to_92.1%25_Savings-06B6D4.svg)](https://tokensafe.dev#benchmarks)
[![Python: 3.10+](https://img.shields.io/badge/Python-3.10+-3B82F6.svg)](https://python.org)
[![Trial: 7 Days Free](https://img.shields.io/badge/Trial-7_Days_Free-10B981.svg)](https://tokensafe.dev#pricing)

> **Autonomous Zero-Loss Token Economy & Context Optimizer for AI Coding Agents.**  
> Slashes LLM token costs by **60%–90%** with zero quality loss. Built for **Google Antigravity, OpenAI Codex, Cursor, Claude Desktop, Z.ai Code, Windsurf & Cline**.

🌐 **Official Website:** [https://tokensafe.dev](https://tokensafe.dev)

---

## ⚡ 1-Click Universal Install

Connect TokenSafe to all detected AI coding agents on your machine in seconds:

```bash
# Option 1: Zero-Install via npx (Recommended for Cursor & Claude Desktop)
npx tokensafe

# Option 2: Python / PyPI
pip install tokensafe && tokensafe

# Option 3: Direct Shell Script (macOS / Linux)
curl -fsSL https://tokensafe.dev/install.sh | bash
```

---

## 🔌 Universal MCP Server Configuration

Connect TokenSafe directly to **Claude Desktop** (`claude_desktop_config.json`) or **Cursor** MCP Settings:

```json
{
  "mcpServers": {
    "tokensafe": {
      "command": "npx",
      "args": ["-y", "tokensafe", "mcp"]
    }
  }
}
```

---

## 🚀 The Breakthrough: How It Saves 60%–90% Tokens

Traditional AI code interactions suffer from massive context bloat:
1. **The BPE Token Penalty:** Verbose natural language prompts and filler waste 40%–70% of ingress context.
2. **Bulk Tool Dumps:** Reading whole 800+ line files burns 5,000+ tokens per turn when only 20 lines are needed.
3. **Stuck Loops & Bloated Diffs:** Terminal `git diff` and repetitive failed commands spiral context into overflow.

### TokenSafe Solves All Three Autonomously:
- **TOCP Prompt Distillation:** Intercepts prompts and distills intent into lean S-Expressions via **Google Gemini (2.0 Flash-Lite), OpenAI ChatGPT (gpt-4o-mini), or DeepSeek-Flash** (`-40% to -70%`). Automatically detects `GEMINI_API_KEY`, `OPENAI_API_KEY`, or `DEEPSEEK_API_KEY`.
- **AST Skeletonization & Slicing:** Folds inactive function bodies into signatures and windows targeted symbols (`-80% to -92%`).
- **Predictive Dependency Prefetcher:** Attaches imported symbol signatures to eliminate downstream exploratory `read_file` roundtrips.
- **Output Guard & Loop Guard:** Blocks destructive full-file rewrites on files $\ge 40$ lines and halts command loops.

---

## 🤖 Universal LLM Distillation Providers

TokenSafe supports any key you already have:

| Provider | Supported Model | Environment Variable | Config Key |
|---|---|---|---|
| **Google Gemini** | `gemini-2.0-flash-lite`, `gemini-2.0-flash` | `GEMINI_API_KEY` | `gemini_api_key` |
| **OpenAI ChatGPT** | `gpt-4o-mini`, `gpt-4o` | `OPENAI_API_KEY` | `openai_api_key` |
| **DeepSeek** | `deepseek-flash`, `deepseek-chat` | `DEEPSEEK_API_KEY` | `deepseek_api_key` |

Check active providers anytime with:
```bash
tokensafe providers
```

---

## 📊 Zero-Loss SWE-bench Benchmark Results

Verified across 5 real-world production scenarios:

| Benchmark Scenario | Task Complexity | Baseline Tokens | TokenSafe Tokens | Reduction | Turn Speedup | Accuracy |
|---|---|---|---|---|---|---|
| **Django ORM N+1 Fix** | Multi-file query refactoring | 14,280 | **1,120** | **-92.1%** | +12% faster | 100% Pass |
| **FastAPI Async Race** | Concurrency bug investigation | 8,640 | **2,940** | **-65.9%** | +10% faster | 100% Pass |
| **TypeScript Redux Vault** | Full AST symbol refactoring | 18,400 | **2,130** | **-88.4%** | +9% faster | 100% Pass |
| **Stuck Command Loop** | Subprocess infinite loop | 22,100 | **2,400** | **-89.1%** | Stopped loop | 100% Pass |
| **Heavy Rewrite Guard** | Whole-file write attempt | 11,300 | **4,310** | **-61.8%** | Forced edit_file | 100% Pass |

---

## 💎 Pricing & 7-Day Free Trial

| Tier | Price | Highlights |
|---|---|---|
| **Hacker (Open-Core)** | **$0** (Free Forever) | 100k tokens/mo, Local Regex compactor, use own DeepSeek key |
| **Solo Developer** | **$9 / month** | **7-Day Free Trial**, Unlimited Managed DeepSeek-V3, AST Prefetcher, Output Guards |
| **Engineering Team** | **$29 / dev / mo** | **7-Day Free Trial**, Team Budget Caps, Centralized Ledger, B2B Docker Proxy |
| **Enterprise** | Custom | On-Premise Nuitka C-compiled Docker Proxy, Custom Compliance, SAML SSO |

👉 **[Start your 7-Day Free Trial on our website](https://tokensafe.dev#pricing)**

---

## 🛠️ CLI Quickstart

```bash
# View live savings dashboard in your terminal
tokensafe stats

# Open interactive Apple dark-mode HTML dashboard
tokensafe html

# Run the 5-scenario benchmark suite
tokensafe benchmark

# Run the local transparent AI proxy (127.0.0.1:8765/v1)
tokensafe proxy

# Activate your Pro License key
tokensafe activate TS1.eyJlbWFpbCI6...
```

---

## 🧪 Unit Testing

```bash
python3 -m unittest discover tests
```
*Zero external runtime dependencies — runs on standard Python 3.10+.*

---

## 📄 License
MIT License. Commercial enterprise licenses available for on-premise B2B deployments.

### Export Markdown Report
```bash
./bin/agy-tokens markdown
```

---

## 📂 Project Structure

```text
agy-token-optimizer/
├── plugin.json                 # Antigravity plugin manifest
├── hooks.json                  # Lifecycle hook declarations (PreInvocation, PreToolUse, PostToolUse)
├── rules/
│   └── AGENTS.md               # TOCP Protocol directives for LLM (prompt-cached)
├── skills/
│   └── token-optimizer/
│       └── SKILL.md            # On-demand Antigravity skill
├── proto/
│   └── tocp.proto              # Protocol Buffers contract
├── src/
│   ├── tokenizer.py            # Zero-dependency calibrated BPE token counter
│   ├── compactor.py            # Semantic compactor (Natural Language -> TOCP S-Expr)
│   ├── slicer.py               # AST & Line windowing tool optimizer
│   ├── metrics.py              # Persistent SQLite metrics engine
│   └── dashboard.py            # Visual ANSI terminal & Markdown generator
├── hooks/
│   ├── pre_invocation.py       # PreInvocation hook entry point
│   ├── pre_tool_use.py         # PreToolUse hook entry point
│   └── post_tool_use.py        # PostToolUse hook entry point
├── bin/
│   └── agy-tokens              # Executable CLI
├── tests/                      # Unit test suite (100% passing)
├── install.sh                  # One-click installer for Antigravity
└── README.md
```

---

## 🛠️ Installation & Antigravity Setup

Run the installer:
```bash
./install.sh
```
This symlinks the plugin directly into `~/.gemini/config/plugins/agy-token-optimizer`.  
Antigravity automatically loads it on startup.

---

## 🧪 Running Unit Tests

```bash
python3 -m unittest discover tests
```
No external packages required — runs on standard Python 3.10+.
