Metadata-Version: 2.4
Name: shimchain
Version: 0.1.0
Summary: Chain of shell-command shims routing calls through semantic tools, compression backends, and real binaries
License: MIT
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# shimchain

A chain of shims that routes AI-agent shell commands through semantic MCP tools, CLI output compression, and hook-based nudging — whichever link can handle the call handles it, otherwise it falls through to the next. One installer wires up three layers covering the three paths an agent can take when interacting with your code.

## Why?

Token-reduction and context-quality tools for AI coding agents fall into two camps:

- **Transparent CLI proxies** like RTK compress noisy command output (git, docker, kubectl, grep, etc.) so it takes fewer tokens in the context window.
- **Semantic MCP servers** like Serena, jCodeMunch, tree-sitter-mcp, and Code Pathfinder use LSP or AST to give agents symbol-level understanding of a codebase instead of raw text reads. Some are read-oriented or specialized; Serena supports edits.

Neither covers the other's ground:
- Serena won't help you with `git status` output.
- RTK won't help an agent understand what a function does or where it's called.

You don't need to choose. This tool glues their strengths together and fills the gaps: when a semantic backend can answer the question, nudge toward it; when it can't, make sure the raw command's output is at least compressed; when the agent uses a Claude Code built-in tool that bypasses both, catch it at the hook layer.

One install command sets up the whole stack, auto-detects what's already installed, and lets power users customize via a single `mappings.json`.

## The three layers

1. **PATH shims** — intercept shell commands (`grep`, `cat`, `sed`, `find`, etc.) when an agent runs them via a Bash/shell tool.
2. **PreToolUse hook** — intercepts Claude Code's built-in tools (`Grep`, `Read`, `Edit`, `Write`, `Glob`), which have their search/edit engines compiled into the claude binary and don't go through PATH.
3. **Compression fallback** — when a wrapper backend (e.g. RTK) is installed, the shim and the escape hatch (`grep_`, `cat_`, etc.) both route through it instead of running the raw binary.

## Coverage matrix

Comparison of the three approaches across the paths an agent can take:

| Scenario | Semantic MCP server alone (e.g. Serena) | Transparent CLI proxy alone (e.g. RTK via `rtk init -g`) | **This tool** |
|---|---|---|---|
| Agent runs `grep` via Bash tool | ❌ by default — no shell-layer interception. 🟡 possible if user manually installs the server's own hook (e.g., Serena ships a remind hook you can wire into settings.json by hand). | ✅ rewrites to `rtk grep` → compressed output, but no nudge toward semantic alternative | ✅ PATH shim intercepts → Serena nudge when backend ready; else routes through rtk (compressed) when installed; else real grep |
| Agent uses built-in `Grep`/`Read`/`Edit` tool | 🟡 some servers (Serena, jcodemunch) ship optional hooks that cover this — but you have to install/configure them yourself by editing settings.json. Not on by default. | ❌ rtk's hook targets `Bash` only — can't reach built-in tools (which don't shell out) | ✅ PreToolUse hook counts consecutive calls, denies with semantic suggestion after threshold — installed and wired up automatically |
| Agent bypasses with explicit escape hatch (`grep_`) | ❌ no escape hatch concept | ❌ no escape hatch concept | ✅ stderr reminder about semantic alt, then routes through `rtk grep` if available for compression |
| User types `grep` in an interactive terminal | ❌ | ❌ — `rtk init -g` only installs a Claude Code hook; it does NOT add shell aliases. Type `rtk grep` directly or add your own alias. | ❌ — same reason. Our PATH only applies inside the AI tool's env. Neither tool covers this path by default. |
| Setup | Install the MCP server; optionally hand-configure its hooks into settings.json | `rtk init -g` (Claude Code hook only) | `pipx install shimchain && shimchain install` — auto-detects backends, installs missing ones, configures shims + hook + PATH |

## Install

Requires Python 3.9+.

```bash
pipx install shimchain        # or: uv tool install shimchain
shimchain install             # interactive: one question for defaults, "custom" for picking
shimchain install -y          # non-interactive, auto-install missing recommended backends
shimchain install --backend serena    # force-enable specific backend(s)
shimchain install --project           # write to .claude/settings.json in cwd instead of ~
shimchain install --no-hooks          # skip the PreToolUse hook
shimchain install --isolate           # separate shim copy per provider
```

Installed shims live in `~/.shimchain/shims/` (materialized from the package on first install). The interactive flow shows what's recommended, what's optional, what each backend does and why, then asks one question. Pick `custom` to choose backends individually.

## Uninstall

```bash
shimchain uninstall                   # remove shims + hooks + PATH from configs, delete ~/.shimchain/
shimchain uninstall --project         # remove project-level install
shimchain uninstall --clean-backends  # also uninstall MCP backend binaries (serena, etc.)
```

## Development

```bash
git clone <repo> && cd shimchain
pip install -e .
python -m pytest test/
```

## How the layers interact at runtime

```
agent calls `grep "x" file.py` via Bash tool
        ↓
    PATH shim fires (gen/grep)
        ↓
    Serena ready (readiness_cmd succeeds)?
    ├─ YES → block, stderr: "use find_symbol instead"
    └─ NO  → RTK installed?
             ├─ YES → exec `rtk grep "x" file.py` (compressed output)
             └─ NO  → exec real `grep "x" file.py` (raw output)

agent calls `grep_ "x" file.py` (escape hatch — explicit bypass)
        ↓
    stderr: "[ESCAPE HATCH] consider find_symbol"
        ↓
    RTK installed? → use it; else real grep

agent uses Claude Code's built-in `Grep` tool
        ↓
    PreToolUse hook fires
        ↓
    tool_name ∈ {grep, read, edit, write, glob}?
    ├─ NO  → no-op (Bash calls fall through here)
    └─ YES → count consecutive calls
             ↓
             Threshold reached (3 same / 4 mixed)?
             ├─ YES → deny with suggestion, reset counter, 2-min cooldown
             └─ NO  → allow, increment
             ↓
             Counter resets when agent uses any mcp__<backend>__* tool
```

## Architecture

```
mappings.json           # human-editable: backends, shims, hook_redirects, workflows
install.py              # generates everything, configures provider, installs hooks
uninstall.py            # reverses install.py, optionally removes backend binaries
test/test_install.py    # pytest suite (105 tests)
shims/
  base.sh               # shared logic: tool-name resolution, real-binary lookup, mappings load
  tool_shim.sh          # blocking shim with compression fallback
  escape_hatch.sh       # always-pass-through with compression preference
  gen/                  # gitignored — regenerated on every install
    tool_mappings.sh    # 6-column pipe-delimited, generated from mappings.json
    pretooluse_hook.py  # generated, self-contained, stateful via JSON file
    grep -> ../tool_shim.sh
    grep_ -> ../escape_hatch.sh
    ...
```

Shim scripts use only bash builtins internally — they never call `grep`/`sed`/`cat`/etc., which would recurse into themselves.

## mappings.json structure

Top-level keys:

- `backends` — MCP servers (or shell wrappers like RTK). Each defines `install_check`, `install_cmd`, `start_cmd`/`readiness_cmd` (for MCP servers) or `wrap_cmd` (for shell wrappers), `recommended`, `category`, `description`, `benefits`, `url`.
- `shims` — bash commands that get a PATH shim. Each can have a primary `backend` (semantic alternative) and/or `compression_backend` (fallback wrapper). `hook_tools` lists Claude Code built-in tools that should also nudge to the same alternative.
- `hook_redirects` — hook-only mappings for tools without a shell command (e.g., `WebSearch` → `exa`).
- `hooks` — thresholds for the PreToolUse hook (consecutive count, combined count, cooldown seconds).
- `workflows` — documentation only: recommended backend combinations for different use cases.

Example shim with both backends:

```json
"grep": {
  "use_instead": ["find_symbol", "find_referencing_symbols"],
  "backend": "serena",            // primary: blocks when ready
  "compression_backend": "rtk",   // fallback: routes through rtk if not blocking
  "hook_tools": ["Grep"]          // also nudge built-in Grep tool via hook
}
```

## Currently configured backends

| Backend | Category | Recommended default | What it does |
|---|---|---|---|
| serena | code-semantic | ✅ | LSP-powered find/read/edit symbols |
| rtk | cli-output | ✅ | Compresses CLI output 60-90% (git, docker, kubectl, grep, cat, …) |
| jcodemunch | code-semantic | | Tree-sitter symbol indexing, 95%+ token reduction (read-only) |
| codepathfinder | code-semantic | | AST call graph, "who calls X" |
| tree-sitter-mcp | code-semantic | | AST queries across 25+ languages |
| exa | web-search | | Real code/web search (GitHub, docs) |
| firecrawl | web-scraping | | JS-rendered scraping → markdown |

**Pick at most one from `code-semantic`** — they overlap. Stack other categories freely.

## Workflows (from `mappings.json`)

- **default** (`serena + rtk`) — best balance for most users
- **minimal** (`serena`) — just code semantics
- **max_token_savings_readonly** (`jcodemunch + rtk`) — read-heavy, no editing
- **call_graph_focused** (`codepathfinder + rtk`) — heavy on tracing relationships
- **polyglot_codebase** (`tree-sitter-mcp + rtk`) — many languages, mixed
- **with_web** (`+exa +firecrawl`) — additive layer for web work

## Adding things to mappings.json

**New shim:**
```json
"shims": {
  "awk": {
    "use_instead": ["get_symbols_in_file"],
    "backend": "serena",
    "compression_backend": "rtk",
    "hook_tools": []
  }
}
```

**New backend (MCP server):**
```json
"backends": {
  "my-lsp": {
    "description": "...",
    "category": "code-semantic",
    "recommended": false,
    "install_check": "command -v my-lsp",
    "install_cmd": "pip install my-lsp",
    "uninstall_cmd": "pip uninstall -y my-lsp",
    "start_cmd": "my-lsp serve",
    "readiness_cmd": "pgrep -f my-lsp",
    "tool_prefix": "mcp__my_lsp__",
    "url": "https://..."
  }
}
```

**New backend (CLI wrapper like RTK):**
```json
"backends": {
  "compresscli": {
    "install_check": "command -v compresscli",
    "wrap_cmd": "compresscli"
  }
}
```

**Hook-only redirect (no shell command involved):**
```json
"hook_redirects": {
  "WebSearch": { "use_instead": ["web_search"], "backend": "exa" }
}
```

Re-run `python3 install.py` after any change.

## Adding a new AI provider

Add to `PROVIDER_CONFIGS` in both `install.py` and `uninstall.py`:

```python
PROVIDER_CONFIGS = {
    ...
    "newprovider": (".newprovider", "newprovider.json"),
}
```

## Tests

```bash
python3 -m pytest test/ -v
```

105 tests covering install/uninstall in shared/isolated/project modes, mappings validation, hook script behavior (counting, cooldown, deny semantics, backend prefix detection), shim blocking/fallthrough, escape hatch behavior, compression layering (semantic-blocks-wins, wrap-fallback, missing-wrap-falls-to-real), missing binaries, PATH recursion avoidance, stdin passthrough, and argument edge cases.

## How this compares to using backends directly

For **Claude Code users**, this tool covers strictly more than Serena alone or RTK alone:
- **Serena alone:** misses Bash-tool commands that bypass its MCP layer (grep, cat, sed via shell); no output compression.
- **RTK alone:** no semantic alternative; no hook coverage for built-in Grep/Read/Edit.
- **This tool:** PATH shims + hook + compression fallback, one install, one config, covers all three paths.

For **non-Claude-Code users** (Copilot, Cursor, Windsurf, etc.), you get the PATH-shim layer only — same coverage as stacking Serena + RTK manually, just with a unified installer. No provider currently exposes a hook API comparable to Claude Code's, so the built-in-tool coverage gap is an ecosystem constraint that affects every alternative equally.

## Things worth knowing

- **`settings.json` `env.PATH` can't reference `$PATH`.** The installer resolves the full path at install time.
- **Running `rtk init -g` in addition to this installer is redundant for Claude Code coverage.** Both install a Claude Code PreToolUse hook that covers Bash tool calls — rtk's hook redirects to `rtk grep` directly, ours routes through `rtk grep` via `wrap_cmd` only when the semantic backend isn't ready. If you install both, rtk's hook fires first and redirects `Bash(grep)` → `rtk grep` *before* our PATH shim sees it, so our Serena nudge for Bash-tool grep is bypassed (you still get the nudge for built-in `Grep` via our separate hook). `rtk init -g` does NOT install shell aliases, so it doesn't help with interactive terminal use either way. Install ordering doesn't matter — our hook no-ops for Bash calls (it only counts built-in tool names), so the two hooks don't interfere with each other in the PreToolUse array.
- **`mappings.json` uses hardcoded tool names** (e.g., `find_symbol`). If an upstream MCP backend renames one of its tools, the `use_instead` hint becomes stale until you update the mapping.
