Metadata-Version: 2.4
Name: hermes-token-diet
Version: 0.2.3
Summary: Universal context-token diet tool for Hermes Agent
Project-URL: Homepage, https://github.com/deresolution20/hermes-token-diet
Project-URL: Issues, https://github.com/deresolution20/hermes-token-diet/issues
Author-email: Brice Neal <brice.neal@proton.me>
License: MIT
License-File: LICENSE
Keywords: agent,context,hermes,llm,tokens
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Utilities
Requires-Python: >=3.10
Requires-Dist: pyyaml>=6.0
Requires-Dist: rich>=13.0
Requires-Dist: ruamel-yaml>=0.18
Provides-Extra: dev
Requires-Dist: bandit>=1.7; extra == 'dev'
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pytest-cov>=4.0; extra == 'dev'
Requires-Dist: pytest-timeout>=2.0; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Provides-Extra: precise
Requires-Dist: tiktoken>=0.7; extra == 'precise'
Provides-Extra: test
Requires-Dist: pytest-cov>=4.0; extra == 'test'
Requires-Dist: pytest>=7.0; extra == 'test'
Description-Content-Type: text/markdown

<div align="center">

# 🫒 Hermes Token Diet

**Cut your Hermes Agent context-window burn by 30%+ — without patching a single line of code.**

[![CI](https://github.com/deresolution20/hermes-token-diet/actions/workflows/ci.yml/badge.svg)](https://github.com/deresolution20/hermes-token-diet/actions/workflows/ci.yml)
[![Release](https://img.shields.io/github/v/release/deresolution20/hermes-token-diet?include_prereleases)](https://github.com/deresolution20/hermes-token-diet/releases)
[![Python](https://img.shields.io/badge/python-3.10%2B-blue.svg)](https://www.python.org/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

</div>

Hermes ships with a kitchen-sink of toolsets and a huge skills catalog. That is great for a first-time user, but for a daily driver the context window fills up fast — and every extra token is a token you pay for. **Hermes Token Diet** is a small, opinionated CLI that audits your real billed `prompt_tokens`, shows you exactly what is burning tokens, applies a safe preset, and proves the savings with a live before/after demo.

> ⚡️ Config-only. Reversible. One-minute setup.

---

## ✨ What it does

| Before | After | Δ |
|---|---|---|
| **~19,200 tokens** just to say `"hi"` | **~13,140 tokens** | **-32%** fixed overhead |
| **~9,500 tokens/exchange** growth | **~371 tokens/exchange** growth | **-96%** per-exchange growth |

Numbers above are from a live `kimi-k2.6` / Ollama Cloud run on Hermes v0.17.0. Your mileage depends on your persona and tool usage, but the tool reports *your* actual numbers every time.

---

## 🚀 Install

### pipx (recommended)

```bash
pipx install git+https://github.com/deresolution20/hermes-token-diet.git
```

Run once without installing:

```bash
pipx run git+https://github.com/deresolution20/hermes-token-diet.git audit
```

### uv

```bash
uv tool install git+https://github.com/deresolution20/hermes-token-diet.git
```

### From source

```bash
git clone https://github.com/deresolution20/hermes-token-diet.git
cd hermes-token-diet
pip install -e ".[dev]"
pytest -q
```

---

## 🎯 Quick start

### 1. Measure your current baseline

```bash
hermes-token-diet audit
```

```text
╭──────────────────────────────────────╮
│ Hermes Token Diet — Fresh 'hi' audit │
╰──────────────────────────────────────╯
Hermes home: /home/brice/.hermes
Running hermes chat -q 'hi' from a neutral directory...

  Metric                                           Value
 ────────────────────────────────────────────────────────
  Session ID                      20260702_090329_26536f
  Input tokens (fixed overhead)                   13,140
  Output tokens                                       73
  Target fixed overhead                         ≤ 15,000
  Status                                 ✅ under target
```

### 2. See what costs tokens

```bash
hermes-token-diet analyze
```

Shows every enabled toolset and skill category with estimated token cost, plus your current `file_read_max_chars` / `tool_output.max_bytes` caps.

### 3. Preview a diet profile

```bash
hermes-token-diet recommend --profile daily
```

### 4. Live before/after demo

```bash
hermes-token-diet demo --profile daily
```

This measures your current cost, applies the profile, measures again, and prints the delta — with an automatic timestamped backup of `~/.hermes/config.yaml`.

### 5. Verify long-session growth

```bash
hermes-token-diet verify
```

Runs a scripted 5-exchange working session and reports average token growth per turn. Target is ≤ 5,000 tokens/exchange.

---

## 👁️ Context Watch addon

Monitor live Hermes sessions for context-window fatigue. Reports metrics, warns when you cross configurable thresholds, and writes reports to `~/.hermes/scratch/context-diet-latest.md` so Hermes can read its own diet advice.

### Manual check

```bash
# Check the active session
hermes-token-diet check

# Check a specific session
hermes-token-diet check --session-id 20260702_090329_26536f

# Show the markdown report in the terminal too
hermes-token-diet check --show-report
```

### Background watch (foreground)

```bash
hermes-token-diet watch --interval 300
```

Polls every 5 minutes and writes a report only when a session crosses from healthy into `warning` or `critical`. Stop with `Ctrl+C`.

### Daemon mode

Run unattended with a PID file and log file:

```bash
# Start the daemon
hermes-token-diet watch --daemon

# Check status
hermes-token-diet watch --status

# Stop it
hermes-token-diet watch --stop

# Opt-in auto-apply: when a session hits critical, automatically apply the
# 'daily' diet profile (with a backup) and keep watching
hermes-token-diet watch --daemon --auto-apply
```

Daemon files live in `~/.hermes/scratch/`:
- `context-watch-default.pid`
- `context-watch-default.log`

### Install as an OS service

```bash
hermes-token-diet watch --install-service
```

This prints copy-and-paste instructions for `systemd --user`, macOS `launchd`, and Windows Task Scheduler using the templates in `extras/`.

### Fine-tune thresholds

Add a `token_diet_watch` block to `~/.hermes/config.yaml`:

```yaml
token_diet_watch:
  thresholds:
    token_ratio_warning: 0.60
    token_ratio_critical: 0.80
    compactions_warning: 3
    compactions_critical: 6
    growth_per_exchange_warning: 4000
    growth_per_exchange_critical: 7000
```

### Re-enable tools or skills on the fly

```bash
# Bring a toolset back
hermes-token-diet enable-tool browser

# Bring a skill back
hermes-token-diet enable-skill ascii-art

# Undo the last config change
hermes-token-diet revert

# List available backups
hermes-token-diet revert --list
```

---

## 📦 Profiles

| Profile | Toolsets kept | Caps | Best for |
|---|---|---|---|
| `minimal` | Core assistant only (`file`, `terminal`, `memory`, `skills`, `todo`, `session_search`, `context_engine`) | 20K file / 8K output | Minimal token budget, headless use |
| `daily` | Core + `web`, `browser`, `code_execution`, `cronjob` | 20K file / 8K output | Coding / ops assistant *(recommended)* |
| `balanced` | Daily + kept `creative` / `media` skills | 30K file / 12K output | Multimedia and research workflows |
| `aggressive` | `file`, `terminal`, `session_search` only | 8K file / 4K output | Very long personas or severe context pressure |

Add `--usage-tuned` to any profile to re-enable toolsets or skill categories you actually used in the last 14 days, pulled from `~/.hermes/state.db`.

---

## 🛡️ How it stays safe

- **No Hermes patches.** Every change is a documented edit to `~/.hermes/config.yaml`.
- **Automatic backups.** `apply`, `demo`, and the TUI create timestamped `.yaml.bak.token-diet-*` files before touching anything.
- **Skills stay loadable.** Disabling a skill only removes it from the eager system-prompt index; `skill_view(name='...')` still works on demand.
- **Input validation.** Paths, labels, and session IDs are validated; backups must live inside `~/.hermes`.
- **Hardened CI.** Every push runs `ruff`, `mypy`, `bandit`, tests, and a wheel smoke-test.
- **MCP servers preserved.** Platform-specific and MCP config is left untouched.
- **Uses the real Hermes venv.** The tool discovers your Hermes Python interpreter and runs probes through it, so measurements match reality.

---

## 🔧 What actually changes in `config.yaml`

```yaml
platform_toolsets:
  cli:
    - file
    - terminal
    - memory
    - skills
    - todo
    - session_search
    - context_engine
    - web
    - browser
    - code_execution
    - cronjob

file_read_max_chars: 20000
tool_output:
  max_bytes: 8000

skills:
  disabled:
    - apple-notes
    - apple-reminders
    - findmy
    - imessage
    # ... heavy / niche skill categories only
```

All changes are scoped, reversible, and tailored to the chosen profile.

---

## 🧪 Development & testing

```bash
# Clone and set up a local dev venv
git clone https://github.com/deresolution20/hermes-token-diet.git
cd hermes-token-diet
python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"

# Run the suite
pytest -q
```

CI runs on Ubuntu, macOS, and Windows for Python 3.10–3.13.

---

## 🎮 Available commands

- `audit` — measure your current fixed overhead
- `analyze` — see what each enabled toolset / skill costs
- `recommend --profile <name>` — preview a diet profile
- `demo --profile <name>` — live before/after benchmark with auto-backup
- `verify` — measure long-session token growth
- `check` / `watch` — monitor live sessions for context-window fatigue
- `diff <a> <b>` — compare two profiles side by side
- `revert` — restore config.yaml from a token-diet backup
- `tui` — interactive terminal UI for browsing profiles and sessions, with one-click Apply/Revert

## 📖 Documentation

- [`docs/token-audit.md`](docs/token-audit.md) — full Phase 1 audit report with counter semantics, fixed-overhead breakdown, and verified Phase 4 results.
- [`workflows/hermes_token_diet.md`](workflows/hermes_token_diet.md) — human SOP for running the tool end to end.
- [`CHANGELOG.md`](CHANGELOG.md) — release notes.
- [`CONTRIBUTING.md`](CONTRIBUTING.md) — how to contribute.

---

## 🗺️ Roadmap

- [x] `check` / `watch` Context Watch addon with live session monitoring
- [x] `revert` command to restore from any timestamped backup
- [x] `diff` command to compare two profiles side by side
- [x] Per-provider token counting with `tiktoken` / local tokenizer fallbacks
- [x] `--aggressive` profile for very long personas
- [x] TUI mode for interactive exploration

---

## 💡 Why context tokens matter

Hermes sends the entire system prompt + full transcript + tool schemas on every API turn. If your fixed overhead is 19K tokens and you grow by ~10K per exchange, a 256K window is half full after only ~23 turns. **Compressing the fixed payload and capping tool-output retention** keeps sessions cheaper, faster, and less likely to hit the context ceiling.

This tool makes that tuning measurable, repeatable, and reversible.

---

## 📄 License

[MIT](LICENSE) © Brice Neal

---

<div align="center">

Built with the <a href="https://hermes-agent.nousresearch.com/">Hermes Agent</a> ecosystem in mind.

</div>
