Metadata-Version: 2.4
Name: hermes-lite
Version: 0.7.0
Summary: Hermes-Lite — local-first AI agent with local Qwen model + NVIDIA NIM cloud escalation. MoA orchestration, streaming, WebUI, rate limiting, sandbox security, standalone web search via ddgs.
Author-email: Ahmed Hassan <a.hassan.b.h@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/ahmedhabibo/hermes-lite
Project-URL: Repository, https://github.com/ahmedhabibo/hermes-lite
Project-URL: Issues, https://github.com/ahmedhabibo/hermes-lite/issues
Project-URL: Changelog, https://github.com/ahmedhabibo/hermes-lite/releases
Keywords: ai,agent,llm,nvidia-nim,moa,mixture-of-agents,llama-cpp,tool-calling
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pydantic>=2.0
Requires-Dist: aiosqlite>=0.20.0
Requires-Dist: prompt-toolkit>=3.0.0
Requires-Dist: rich>=13.0.0
Requires-Dist: openai>=1.40.0
Requires-Dist: fastapi>=0.110.0
Requires-Dist: uvicorn>=0.27.0
Provides-Extra: stripe
Requires-Dist: stripe>=7.0.0; extra == "stripe"
Provides-Extra: test
Requires-Dist: pytest>=7.0; extra == "test"
Requires-Dist: pytest-asyncio>=0.21.0; extra == "test"
Dynamic: license-file

# Hermes-Lite ⚡

[![Tests](https://img.shields.io/badge/tests-467%20passing-brightgreen)](./tests)
[![Python 3.9+](https://img.shields.io/badge/python-3.9%2B-blue)](./pyproject.toml)
[![License: MIT](https://img.shields.io/badge/license-MIT-yellow)](./LICENSE)
[![Cloud: NVIDIA NIM](https://img.shields.io/badge/cloud-NVIDIA%20NIM-76B900)](https://build.nvidia.com/)

> **Cloud-first AI agent via NVIDIA NIM Free API — with local fallback for offline use.**

Hermes-Lite is an agent framework that defaults to cloud LLMs (NVIDIA NIM Free API) for quality, with a local mode for offline/privacy use. Rate limiting, API key rotation, and a smart fallback chain keep it resilient — all for free.

---

## Why Hermes-Lite?

- ☁️ **Cloud-first, free tier:** NVIDIA NIM Free API gives 40 RPM of production-grade LLMs (z-ai/glm-5.2, Kimi K2.6, Qwen 3.5, DeepSeek V4) at zero cost
- 🔄 **Resilient by default:** Token-bucket rate limiter, exponential backoff on 429s, API key rotation from a pool
- 🏠 **Local fallback:** Switch to a local 7B Qwen model for offline/privacy — same codebase, different prefix
- 🛡️ **Smart routing:** Complexity-based tier selection. Simple queries → fast model. Complex reasoning → heavy model. Escalation on repeated failures.
- 🧩 **6 built-in tools:** `read_file`, `search_files`, `terminal`, `memory`, `web_search`, `web_fetch`
- 🔌 **Sub-agent delegation:** Spawn isolated subagents for parallel work

---

## Quick Start

### Cloud Mode (default, no local model needed)

```bash
# 1. Set your NVIDIA NIM API key (free at https://build.nvidia.com/)
export HERMES_LITE_NVIDIA_API_KEY="nvapi-..."

# Optional: add multiple keys for rotation
# export HERMES_LITE_NVIDIA_API_KEYS="key1,key2,key3"

# 2. Install hermes-lite
git clone https://github.com/ahmedhabibo/hermes-lite.git
cd hermes-lite
pip install -e ".[test]"

# 3. Run the CLI — starts with cloud models
python -m hermes_lite
```

### Local Mode (offline / privacy)

```bash
# 1. Install llama.cpp
brew install llama.cpp

# 2. Download the model (Qwen 2.5 Coder 7B Instruct, IQ3_XS — 3.1 GB)
hf download bartowski/Qwen2.5-Coder-7B-Instruct-IQ3_XS-GGUF \
    Qwen2.5-Coder-7B-Instruct-IQ3_XS.gguf \
    --local-dir ~/.hermes_lite/models/

# 3. Start the server
llama-server \
    -m ~/.hermes_lite/models/Qwen2.5-Coder-7B-Instruct-IQ3_XS.gguf \
    --port 8080 --temp 0.3 --repeat-penalty 1.1 \
    -ngl 28 -c 65536 --batch-size 512 --cache-type-k q8_0 --cache-type-v q8_0

# 4. Set a local-first fallback chain (or use local: prefix in chat)
export LITE_FALLBACK_CHAIN="local:Qwen2.5-Coder-7B-Instruct-IQ3_XS.gguf,z-ai/glm-5.2,minimaxai/minimax-m3,moonshotai/kimi-k2.6,qwen/qwen3.5-397b-a17b,deepseek-ai/deepseek-v4-flash"

pip install -e ".[test]"
python -m hermes_lite
```

---

## How It Works

```
┌─────────────────────────────────────────────────┐
│              Router (cloud-first)                │
│  LiteRouter: complexity → pick NIM model in chain│
│  Escalation: failure → next model in chain       │
└──────────────┬──────────────────────────────────┘
               │
┌──────────────▼──────────────────────────────────┐
│                  LLM Layer                       │
│  NVIDIA NIM (cloud)   │  Qwen 7B (local)       │
│  40 RPM rate limit    │  Text-parsed tool calls │
│  Key rotation + retry │  No tools/grammar sent  │
└──────────────┬──────────────────────────────────┘
               │
┌──────────────▼──────────────────────────────────┐
│               Tool Loop                          │
│  2-tier: LLM → tool → result → LLM → response   │
│  Max 4 iterations, repeated-error, malformed-JSON│
└──────────────┬──────────────────────────────────┘
               │
┌──────────────▼──────────────────────────────────┐
│           Tool Layer (PluginRegistry)            │
│ read_file │ search_files │ terminal │ memory │  │
│ web_search │ web_fetch │ subagent               │
└──────────────┬──────────────────────────────────┘
               │
┌──────────────▼──────────────────────────────────┐
│              Sandbox / Backend                   │
│  sandbox-exec   │  Hermes MCP   │  curl          │
└──────────────────────────────────────────────────┘
```

---

## Rate Limiting & Key Rotation

Hermes-Lite v0.4+ handles NVIDIA NIM Free API limits automatically:

|| Feature | Detail |
||---------|--------|
|| **Rate limiter** | Token-bucket, 40 RPM (configurable via `HERMES_LITE_RPM`) |
|| **Exponential backoff** | 429 errors: 1s → 2s → 4s → 8s (max 16s) |
|| **Key rotation** | Comma-separated pool in `HERMES_LITE_NVIDIA_API_KEYS`. On 401/403/429, rotates to next key |
|| **Key cooldown** | Failed keys cool down for 60s before reuse |
|| **Max retries** | 4 attempts (configurable via `HERMES_LITE_MAX_RETRIES`) |

---

## NIM Fallback Chain

The default chain (cloud-first):

```
z-ai/glm-5.2           ← preferred (general-purpose)
  → minimaxai/minimax-m3 ← strong reasoning
    → moonshotai/kimi-k2.6 ← MoE, efficient
      → qwen/qwen3.5-397b-a17b ← fast fallback
        → deepseek-ai/deepseek-v4-flash ← ultra-fast
```

Override with `LITE_FALLBACK_CHAIN` env var (comma-separated).

For local-first: `local:Qwen2.5-Coder-7B-Instruct-IQ3_XS.gguf,z-ai/glm-5.2,minimaxai/minimax-m3`

---

## Features

|| Area | What it does |
||------|-------------|
|| **Tool registry** | 6 built-in essentials: `read_file`, `search_files`, `terminal`, `memory`, `web_search`, `web_fetch`. Pydantic-validated dispatch. Extensible via `ToolDefinition`. |
|| **LLM layer** | OpenAI-compatible chat API. Default: cloud NIM (z-ai/glm-5.2). Supports local fallback (Qwen 2.5 Coder 7B via llama.cpp). Rate limiting + key rotation + exponential backoff. |
|| **Tool loop** | Two-tier loop: LLM calls tools → results fed back → LLM responds. Max 4 iterations, repeated-error and malformed-JSON guards. |
|| **Router** | LiteRouter classifies prompts by complexity. Cloud-first chain: light queries → fast model. Complex reasoning → heavier model. Consecutive-failure escalation walks the chain. |
|| **Sandbox** | `terminal` tool runs commands in a macOS sandbox (`sandbox-exec`). Timeout-safe with process lifecycle management. **Sandbox security: command allowlist/blocklist, secret env scrubbing, audit log redaction** |
|| **Memory** | SQLite bridge for cross-session facts. `add`, `replace`, `remove`, `list` — unique-match semantics. Loaded into every prompt (800 char cap). |
|| **Sub-agent** | Spawn isolated subagents for parallel work (default: cloud NIM flash model). Nested orchestration (max 2 levels). **Subagent isolation: env sanitization** |
|| **MoA** | Mixture-of-Agents: run 3–5 diverse LLMs in parallel, then aggregate into a single superior answer. 5 built-in presets (`council`, `speed`, `verification`, `coding`, `creative`). CLI: `/moa <preset>`. |
|| **Observability** | Per-turn JSONL logging, rotation at 10 MB, `python -m hermes_lite.stats` for session summary. |
|| **CLI** | prompt_toolkit + Rich terminal. Ctrl+C/D, `!tool {args}` direct invocation, `/tools`, `/history`, `/help`. |

---

## Configuration

|| Variable | Default | Purpose |
||----------|---------|---------|
|| `HERMES_LITE_CLOUD_URL` | `https://integrate.api.nvidia.com/v1` | Cloud endpoint |
|| `HERMES_LITE_CLOUD_MODEL` | `z-ai/glm-5.2` | Default cloud model |
|| `HERMES_LITE_NVIDIA_API_KEY` | — | Single NVIDIA NIM API key |
|| `HERMES_LITE_NVIDIA_API_KEYS` | — | Comma-separated key pool for rotation |
|| `HERMES_LITE_RPM` | `40` | Requests per minute (token bucket) |
|| `HERMES_LITE_MAX_RETRIES` | `4` | Max retry attempts on 429/server errors |
|| `HERMES_LITE_LOCAL_URL` | `http://127.0.0.1:8080/v1` | Local llama.cpp endpoint |
|| `HERMES_LITE_LOCAL_MODEL` | `Qwen2.5-Coder-7B-Instruct-IQ3_XS.gguf` | Local model file |
|| `HERMES_LITE_LOCAL_TOOLS` | unset | Set `1` to send `tools`/`tool_choice` to local endpoint |
|| `LITE_LOCAL_MAX_COMPLEXITY` | `0.3` | Max complexity score for using lighter model |
|| `HERMES_LITE_MOA_PRESET` | — | Auto-activate MoA preset on startup (e.g. `council`) |
|| `HERMES_LITE_MOA_TIMEOUT` | `30` | Seconds before a reference model is skipped |
|| `LITE_FALLBACK_CHAIN` | `z-ai/glm-5.2,minimaxai/minimax-m3,moonshotai/kimi-k2.6,qwen/qwen3.5-397b-a17b,deepseek-ai/deepseek-v4-flash` | Model fallback chain |
|| `HERMES_LITE_SUBAGENT_MODEL` | `deepseek-ai/deepseek-v4-flash` | Subagent default model |

---

## Mixture-of-Agents (MoA)

Run multiple diverse LLMs on the same prompt in parallel, then let an **aggregator** model synthesize their outputs into a single, superior answer.

### Quick Start

```
❯ /moa council      # Activate the "council" preset (3 diverse refs + 1 aggregator)
❯ Tell me about Python GIL
[MoA] 🔀 3 refs → aggregator → final
```

### Built-in Presets

|| Preset | References | Aggregator | Use case |
||--------|-----------|------------|----------|
|| `council` | z-ai/glm-5.2, minimax-m3, kimi-k2.6 | deepseek-v4-pro | General reasoning — maximum diversity |
|| `speed` | z-ai/glm-5.2, deepseek-v4-flash | deepseek-v4-flash | Fast answers — lighter models |
|| `verification` | kimi-k2.6, qwen3.5-397b, deepseek-v4-pro | z-ai/glm-5.2 | Fact-checking — cross-verify claims |
|| `coding` | deepseek-v4-pro, qwen3.5-397b | deepseek-v4-pro | Code generation — precision-focused |
|| `creative` | z-ai/glm-5.2, kimi-k2.6, qwen3.5-122b | z-ai/glm-5.2 | Creative writing — divergent styles |

### Commands

|| Command | Action |
||---------|--------|
|| `/moa` | Show status + available presets |
|| `/moa council` | Activate a preset |
|| `/moa off` | Deactivate (back to normal ToolLoop) |

### Architecture

```
User prompt
    │
    ├─→ Reference A (model-1) ──┐
    ├─→ Reference B (model-2) ──┼─→ Aggregator ─→ Final answer
    └─→ Reference C (model-3) ──┘
```

- **Parallel mode** (default): `asyncio.gather` runs all references concurrently, then feeds collected outputs to the aggregator
- **Sequential mode**: Each reference sees the previous reference's output as context — useful for iterative refinement
- **Graceful degradation**: If some references fail/timeout, the aggregator works with whatever succeeded. If all fail, falls back to a direct single-model call.

### Environment Variable Override

```bash
HERMES_LITE_MOA_PRESET=council  # Auto-activate on startup
```

---

## Test Suite

```bash
cd hermes-lite
pip install -e ".[test]"
python -m pytest tests/ -v
```

Tests cover: registry (48), memory (47), orchestrator (31), tool loop (15), tools-essentials (55), LLM (5), router (37), sandbox (60), sub-agent (19), memory bridge (10), observability (6), e2e smoke (5), moa 15, api_key_exhaustion 5, sanitize ~20, cli_commands ~10, streaming 4, conftest 1. **467 total.**

---

## Contributing

See [CONTRIBUTING.md](./CONTRIBUTING.md) — PRs welcome, TDD preferred.

---

## License

[MIT](./LICENSE) — use it, fork it, ship it.

---

## CHANGELOG

### 0.6.0 — Security Hardening + Streaming + Docker
2026-07-03
- **Security (Phase 1, 7 items):** API key exhaustion handling, Auth/Authorization framework, Input sanitization pipeline, Rate-limit hardening with jitter, Sandbox tightening (command allowlist/blocklist, secret scrubbing, audit log redaction), Secret redaction in logs, Subagent isolation (env sanitization)
- **Streaming:** `chat_stream()` async generator — token-by-token streaming for both cloud and local endpoints
- **Docker:** Dockerfile + docker-compose.yml for containerized deployment
- **Changed:** Default model to z-ai/glm-5.2, local model to Qwen2.5-Coder-7B-Instruct-IQ3_XS (3.1GB, Bartowski quant)
- **CI:** GitHub Actions — Python 3.9/3.11/3.12 matrix, pytest + coverage + ruff lint
- **Tests:** 432 → 467

### 0.5.0 — Toolchain Expansion + Observability
2026-06-15
- **Added:** web_fetch tool, memory bridge observability, sandbox-exec timeout propagation, subagent nesting limit (2 levels)
- **Changed:** Tool loop max iterations increased to 4, MoA preset verification added, local model quantized to IQ3_XS
- **Tests:** 342 → 432

### 0.4.0 — Cloud-first NIM pivot + rate limiting
- **Cloud-first default:** NIM Free API as primary LLM (MiniMax M3, Kimi K2.6, Qwen 3.5, DeepSeek V4 Flash)
- **Rate limiting:** Token-bucket at 40 RPM (NIM Free API limit)
- **API key rotation:** Comma-separated pool (`HERMES_LITE_NVIDIA_API_KEYS`), auto-rotate on 401/403/429
- **Exponential backoff:** 1s → 2s → 4s → 8s (cap 16s) on 429/server errors
- **Router v2:** Cloud-first fallback chain, consecutive-cloud-failure escalation walks the chain
- **Subagent cloud default:** `deepseek-ai/deepseek-v4-flash` (set `HERMES_LITE_SUBAGENT_MODEL` for local)
- **Local still supported:** `local:` prefix or local-first fallback chain for offline/privacy use
- Added `stepfun-ai/` to cloud prefix detection
- Bare model IDs containing `/` now route to cloud automatically

### 0.3.0 — Qwen 2.5 7B Instruct + text-based tool-call parser
- Upgraded local model: Qwen 2.5 3B → Qwen 2.5 7B Instruct Q4_K_M (4.4 GB)
- Text-based tool-call parser: 4 regex patterns (Qwen blank-line JSON, fenced `tool_call`, fenced JSON, bare JSON)
- Skip `tools`/`tool_choice` for local endpoint — avoids PEG grammar 500 errors on small models
- Cloud fallback via NVIDIA NIM (configurable model chain)
- MIT license + CONTRIBUTING guide added

### 0.2.0 — Local Qwen 3B + 6 essential tools + router + sandbox
- Tool registry with 6 essentials (read_file, search_files, terminal, memory, web_search, web_fetch)
- LiteRouter: prompt complexity classifier for local/cloud tier routing
- ToolLoop: two-tier tool-calling loop with termination guards
- Sandboxed terminal execution (macOS sandbox-exec)
- Memory Bridge: cross-session persistent facts (SQLite)
- Subagent: parallel tool-spawning with isolated context
- Observability: per-turn JSONL logging + stats CLI
