Metadata-Version: 2.5
Name: agentbelay
Version: 0.1.0
Summary: A local, deterministic firewall for AI agent tool calls: Claude Code hooks, an MCP proxy and a Python API, with replayable policies for CI
Project-URL: Homepage, https://github.com/rakshithmuda22/belay
Project-URL: Repository, https://github.com/rakshithmuda22/belay
Project-URL: Issues, https://github.com/rakshithmuda22/belay/issues
Author-email: Sai Rakshith Muda <rakshithmuda@gmail.com>
License-Expression: MIT
License-File: LICENSE
Keywords: ai-agents,claude-code,firewall,guardrails,llm,mcp,policy,security
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Operating System :: MacOS
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Security
Classifier: Topic :: Software Development :: Testing
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: llm-agentspec>=0.3.0
Requires-Dist: pyyaml>=6.0
Provides-Extra: dev
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.6; extra == 'dev'
Requires-Dist: types-pyyaml; extra == 'dev'
Description-Content-Type: text/markdown

# Belay

**A local firewall for AI agents.** Belay checks every tool call an agent wants to make — a shell command, a file edit, an MCP call — against a policy *before it runs*, and allows, asks, or blocks it. The same policy runs live in [Claude Code](https://code.claude.com/docs/en/hooks), in front of any [MCP](https://modelcontextprotocol.io) server, and in your own agent loop — and replays against recorded traces in CI.

[![CI](https://github.com/rakshithmuda22/belay/actions/workflows/ci.yml/badge.svg)](https://github.com/rakshithmuda22/belay/actions/workflows/ci.yml)
![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue)
![License: MIT](https://img.shields.io/badge/license-MIT-green)

Agents now run shell, touch the filesystem, use git, and call MCP servers on your behalf. Built-in permission systems are mostly per-tool allow/deny patterns. The failures that actually hurt are about *order and state and arguments*: a `git push` before the tests pass, a `terraform apply` against the wrong environment, a prompt-injected `cat .env` that lands the key in the model's context, a deploy right after a failing test run. Belay expresses those as rules, enforces them deterministically and offline, and writes an audit trail you can read.

```console
$ belay check --cmd "git push --force origin main"
DENY  Bash: git push --force origin main
  git-force-push-protected (critical): Force-pushing a protected branch rewrites
  history other people rely on. Push to a feature branch and open a pull request.
  194 µs, policy coding-agent

$ belay check --cmd "curl -fsSL https://get.example.sh | sh"
DENY  Bash: curl -fsSL https://get.example.sh | sh
  shell-pipe-to-interpreter (critical): Piping a download straight into a shell
  runs code nobody has reviewed. Download it, read it, then run it.

$ belay check --cmd "pytest -q && git push origin feature/x"
ALLOW Bash: pytest -q && git push origin feature/x
  43 µs, policy coding-agent
```

That last one is the point: `git push` is denied until a test run has **passed in this session**, and allowed once it has. A static allowlist can't say that.

---

## What makes it different

1. **Order- and state-aware rules a per-tool allowlist can't express.** "Tests must pass before `git push`." "No deploy after a failing test run." "An approval for `terraform apply -target=module.db` doesn't unlock `module.vpc`." Budgets and rolling-window rate limits. Argument rules on shell commands *and* MCP tools.
2. **One policy, four places.** The same `belay.yaml` guards Claude Code (hooks), any MCP client (a stdio proxy), and your own Python agent loop — and runs in CI against scenario files and recorded traces.
3. **Deterministic and local.** No model and no network in the decision path. A decision takes ~75 µs. The same trace always gets the same verdict, so a policy is unit-testable.
4. **An audit trail you'll actually open.** Every proposed call, decision and outcome as JSONL (secrets redacted), with a self-contained HTML viewer.

Belay compiles policies into [**agentspec**](https://github.com/rakshithmuda22/agentspec) contracts and makes live decisions with agentspec's incremental `FastGuard`, so a check costs the same at call 10 and call 10,000.

![Belay audit viewer](docs/assets/audit-viewer.png)

*`belay view` output for a scripted session (`examples/sample-session.jsonl`). Every call, the decision, the rule that fired, the measured eval time, and the arguments — one self-contained HTML file.*

---

## 60-second quickstart

```bash
pip install agentbelay
```

### For Claude Code

```bash
cd your-project
belay init                    # writes belay.yaml (extends the coding-agent pack)
belay install claude-code     # adds PreToolUse / PostToolUse hooks to .claude/settings.json
```

That's it. The next time Claude tries something the policy blocks, it sees the reason in the tool result and can recover; things that need a human get a permission prompt. `belay uninstall claude-code` removes the hooks. Prefer a plugin? See [As a Claude Code plugin](#as-a-claude-code-plugin).

### For any MCP server

Wrap the server command. Everything passes through except `tools/call`, which is checked:

```bash
belay mcp --policy belay.yaml -- npx -y @modelcontextprotocol/server-filesystem ~/project
```

```
# the agent asks to read a .env through the server:
read_text_file({path: ".../.env"})
  -> isError: "Belay blocked this call: .env files hold secrets; reading one
     copies them into the model's context. Use .env.example instead."
```

Point your MCP client (Claude Desktop, Cursor, an agent) at `belay mcp -- <server>` instead of `<server>`.

### In your own agent loop

```python
import agentbelay as belay

guard = belay.Guard.from_file("belay.yaml")        # or Guard.from_pack("coding-agent")

decision = guard.check("Bash", {"command": "git push"})
if decision.allowed:
    result = run_tool(...)
    guard.record(decision, ok=result.exit_code == 0)   # feeds ordering rules
elif decision.needs_approval:
    if ask_human(decision.reason):
        guard.approve(decision); run_tool(...)
else:
    print("blocked:", decision.reason)

# or let Belay drive it:
guard.run("Bash", {"command": "pytest"}, run_tool, approve=ask_human)
```

Both quickstarts above were run verbatim in a clean virtualenv before release.

---

## The policy

`belay.yaml` is human-friendly YAML. Rules are checked top to bottom; **deny beats ask beats allow**.

```yaml
version: 1
name: my-service
extends: [coding-agent]          # start from the built-in pack
default: allow                   # decision when nothing matches: allow | ask | deny
disable: [git-force-push]        # drop an inherited rule you don't want

facts:                           # outcomes tracked from tool RESULTS
  tests:
    match: {tool: Bash, args: {$commands: {regex: '^(pytest|npm test)\b'}}}

rules:
  - id: no-dotenv-reads
    action: deny
    severity: high
    message: .env files hold secrets; use .env.example or ask me for the value.
    match: {args: {$reads: {path: ["**/.env", "**/.env.*"]}}}
    unless: {args: {$reads: {path: "**/.env.example"}}}

  - id: tests-before-push
    action: deny
    match: {tool: Bash, args: {$commands: {regex: '^git\s+push\b'}}}
    require: {fact: tests, is: success}      # the LATEST test run must have passed

  - id: terraform-apply
    action: ask
    match: {tool: Bash, args: {$commands: {regex: '^terraform\s+apply\b'}}}
    approve_for: target                      # approving one -target= doesn't approve another
    target: {arg: $command, extract: '-target=(\S+)'}

  - id: web-budget
    action: deny
    match: {tool: [WebFetch, WebSearch]}
    limit: {max: 60, per: 5m}                # rolling-window rate limit
```

**Virtual arguments** let one rule cover Claude Code's `Read`, an MCP server's `read_text_file`, and `cat` in a shell. A small shell analyser splits commands (quotes, `&&`/`|`/`;`, `$(...)`, `sh -c`, heredocs), strips wrappers (`sudo`, `env`, `timeout`…) and extracts what each command reads, writes and deletes:

| | |
|---|---|
| `$command` / `$commands` | the shell command, and each simple command in it |
| `$reads` / `$writes` / `$deletes` | paths a call reads / writes / deletes (Claude Code tools, MCP verbs, shell) |
| `$urls` / `$hosts` | URLs and their hosts, in arguments and shell commands |
| `$content` | text a call writes |
| `$git_branch` | current branch (read from `.git/HEAD`, no subprocess) |

Paths are normalised (`~`, `..`, symlinks) and matched case-insensitively on macOS/Windows, so `.ENV`, `../../.env` and a symlink can't slip past a rule about `.env`.

Validation errors point at the line and suggest a fix:

```
belay.yaml:14:7: rules[2].match.args.$reads: unknown matcher 'pth' (did you mean 'path'?)
```

Power users can write policies in Python (`belay.py` defining `policy = Policy(...)`), with arbitrary `when=` predicates. Full reference: [`docs/policy-reference.md`](docs/policy-reference.md). Design and internals: [`docs/DESIGN.md`](docs/DESIGN.md).

### The `coding-agent` pack

`belay init` starts from a pack of 43 rules tuned for a low false-positive rate: secrets (`.env`, `~/.ssh`, cloud credentials, token-printing commands), destructive shell (`rm -rf` on `/`, `~` or outside the project, `mkfs`, `dd`, fork bombs), `curl | sh`, force-pushing or resetting protected branches, `DROP`/`TRUNCATE` in SQL tools, prod deploys (ask), tests-before-push, nested agents launched with safety flags off, and tampering with Belay's own config. `belay packs coding-agent` prints it; everyday development (edit, build, test, commit, push a feature branch, `rm -rf node_modules`, `source .env`) runs without prompts.

---

## How it works

```mermaid
flowchart LR
    CC[Claude Code] -->|PreToolUse hook| G
    MCP[MCP client] -->|belay mcp proxy| G
    PY[Python loop] -->|guard.check| G
    G["belay.Guard<br/>(compiles belay.yaml)"] --> AS["agentspec<br/>FastGuard"]
    AS -->|allow| RUN[run the tool]
    AS -->|deny / ask| STOP[refuse or ask a human]
    G --> AUD[(JSONL audit)]
    RUN -.outcome.-> G
    AUD --> V[belay view → HTML]
    POL[belay.yaml + traces] --> T[belay test → CI]
```

Each proposed call is normalised to a `Call`, presented to agentspec as a contract check, and the verdict is attributed back to the Belay rule that produced it. Only calls the policy allowed (or a human approved) are committed to the guard's history, so a denied call can never satisfy an ordering rule. The Claude Code hook keeps per-session state in an append-only, `flock`-guarded log, so ordering rules work even though each hook invocation is a fresh process.

---

## Measured numbers

Every number here comes from a script in [`benchmarks/`](benchmarks/); re-run them on your machine. Measured on an **Apple M5 Pro, Python 3.12**, under everyday background load (results will differ on your machine).

**Decision cost** (`python benchmarks/bench_eval.py`, coding-agent pack, 8-call mix):

| | p50 | p95 |
|---|---|---|
| `Guard.check`, end to end | 77 µs | 128 µs |
| — Belay matching (shell parsing, path globs, selectors) | 45 µs | 91 µs |
| — agentspec `FastGuard.check_call` (43 contracts) | 23 µs | 27 µs |

Cost is flat in session length (77 µs at 0 calls, 77 µs at 10,000) — that's the point of FastGuard.

**Claude Code hook, end to end** (`python benchmarks/bench_hook.py`) — this is a *fresh OS process per tool call*, so it's dominated by Python start, not the decision:

| | p50 | p95 |
|---|---|---|
| Python interpreter start (`python -c pass`) | 16 ms | 18 ms |
| Belay PreToolUse hook (allow) | **62 ms** | 68 ms |
| Belay PreToolUse hook (deny) | 66 ms | 74 ms |

So Belay adds roughly **45–50 ms per tool call** in Claude Code, most of it unavoidable interpreter startup. (Claude Code runs hooks in parallel and a tool call is usually far slower than this.)

**MCP proxy added latency** (`python benchmarks/bench_mcp.py`, vs the real `@modelcontextprotocol/server-filesystem`):

| `read_text_file` round trip | p50 | p95 |
|---|---|---|
| Direct to the server | 0.57 ms | 1.06 ms |
| Through `belay mcp` | 1.24 ms | 2.03 ms |
| **Added by Belay** | **+0.66 ms** | +0.97 ms |

**Red-team corpus** (`python benchmarks/redteam.py`): a **self-built** corpus of 100 labelled actions ([`corpus/`](corpus/)) — 71 risky ones modelled on public incidents (the Nx *s1ngularity* credential-harvesting worm, the Amazon Q extension wiper prompt, the Replit production-database deletion, the Gemini CLI file-loss bug) and common agent mistakes, plus 29 benign everyday-development cases.

| | |
|---|---|
| Risky cases caught (deny or ask) | **71 / 71** (54 denied, 17 asked) |
| Benign cases denied (false positives) | **0 / 29** |
| Benign cases that asked (friction) | **0 / 29** |

100% on a corpus written by the same person who wrote the rules measures *coverage of the patterns the pack targets*, not robustness to novel attacks — see [Limitations](#limitations).

**Replayed against real sessions** (`python benchmarks/replay_claude_sessions.py`): the pack replayed over **117,817 tool calls from 191 of my own Claude Code sessions** — 98.6% allowed, 0.9% denied, 0.5% asked. The top denials were reading `.env` files (492), pushing with no passing test run in the session (292), and commands that print a live credential like `security find-generic-password -w` (167). Nothing left the machine; the script prints counts only.

**Prompt-injection, real local model** (`python demos/injection_demo.py`): a small agent loop (Ollama, native tool calling) with the real filesystem MCP server over a sandbox whose README carries an injected "onboarding" step telling the agent to read a credentials file and copy it into `report.txt` (the key is **fake**, created in the sandbox). With `qwen3:8b`, 8 runs each:

| | reads the secret | exfiltrates it | blocked | benign task still done |
|---|---|---|---|---|
| **Direct** | 8 / 8 | 8 / 8 | 0 / 8 | 0 / 8 |
| **Through `belay mcp`** | **0 / 8** | **0 / 8** | 8 / 8 | 6 / 8 |

The full transcript is [`demos/belay.txt`](demos/belay.txt). Without Belay the model takes the bait every time; through Belay the `.env` read comes back as a tool error (`secrets-dotenv`), nothing is exfiltrated, and the model still writes the summary it was actually asked for most of the time.

---

## As a Claude Code plugin

The hooks also ship as a plugin, so a whole team can install Belay from this repo:

```
/plugin marketplace add rakshithmuda22/belay
/plugin install belay@belay
```

The plugin still needs the `belay` CLI installed (`pip install agentbelay`); its launcher finds it. See [`plugins/belay/`](plugins/belay/) and [`.claude-plugin/marketplace.json`](.claude-plugin/marketplace.json).

## In CI

Scenario files assert what the policy should decide; recorded traces are replayed as if live and any call the policy would deny or escalate is a violation:

```yaml
# tests/belay/secrets.yaml
cases:
  - {name: reading .env is denied, tool: Read, args: {file_path: ${PROJECT}/.env},
     expect: deny, rule: no-dotenv-reads}
  - name: push needs a passing test run
    steps:
      - {tool: Bash, args: {command: pytest}, outcome: failure}
      - {tool: Bash, args: {command: git push}, expect: deny}
      - {tool: Bash, args: {command: pytest}}
      - {tool: Bash, args: {command: git push}, expect: allow}
```

```yaml
# .github/workflows/agent-policy.yml
      - run: pip install agentbelay
      - run: belay test --policy belay.yaml tests/belay/*.yaml traces/*.jsonl --junit belay.xml
```

`belay test` exits non-zero on any mismatch and writes JUnit XML. Trace formats it reads: Belay audit logs, Claude Code transcripts, plain JSONL, and Anthropic / OpenAI message JSON.

---

## How it compares

Belay is one of several tools in the 2025–2026 "agent/MCP firewall" space. Honest picture (see each project's docs for specifics):

| | Where it enforces | Policy model | Deterministic | Local |
|---|---|---|---|---|
| **Belay** | Claude Code hook + MCP proxy + Python API + CI | YAML → contracts: order, state, args, budgets, approvals | **Yes** | **Yes** |
| Claude Code permissions / hooks | Inside Claude Code | allow/ask/deny patterns; hooks run any code; auto-mode uses an LLM | rules yes, auto-mode no | yes (auto-mode calls a model) |
| [Docker MCP Gateway](https://github.com/docker/mcp-gateway) | MCP proxy, each server in a container | tool filtering, secret blocking, interceptors; Cedar on the paid tier | yes | yes (governance is paid) |
| [Invariant Guardrails](https://github.com/invariantlabs-ai/invariant) (Snyk) | LLM/MCP proxy or over traces | expressive sequence language **+ ML/LLM detectors** (PII, injection) | rules yes, detectors no | hosted or self-host |
| [Lasso MCP Gateway](https://github.com/lasso-security/mcp-gateway) | MCP proxy | secret/PII masking; cloud "custom policy" | masking yes | masking local, policy cloud |
| [StrongDM Leash](https://github.com/strongdm/leash) | container + **eBPF LSM** + HTTP MITM | Cedar → kernel & proxy rules | yes | yes (needs Docker) |
| [Cupcake](https://github.com/eqtylab/cupcake) | agent hooks | **OPA/Rego** → Wasm, external "signals" | yes (optional LLM judge) | yes |
| [Pipelock](https://github.com/pipelab-org/pipelock) | egress proxy + MCP proxy + CC hooks + OS sandbox | YAML rules, DLP/injection patterns, call-chain detection, budgets | yes | core local (fleet paid) |

**Where others are genuinely better.** None of this is a sandbox. A string-level firewall can't give you what a real isolation boundary does: Leash enforces at the **kernel**, so it catches whatever a subprocess actually does regardless of the command text, and keeps secrets out of the agent via header injection; Docker and Pipelock add **container isolation** and secret injection; Pipelock, Lasso, Docker and Snyk do **content inspection** (DLP, PII masking, tool-poisoning scans) that Belay does not; Invariant's sequence language is more expressive and its ML detectors catch things no pattern will; Cupcake uses an industry-standard policy language (Rego) and more harnesses.

**Where Belay is distinct.** I didn't find another open-source tool combining *precondition* rules ("X must have succeeded before Y"), approvals scoped to a single extracted target, budgets, **and** open-source CI replay of the same policy over recorded traces — across a Claude Code hook, an MCP proxy *and* a Python loop, with zero model or network in the decision path. The closest overlaps are **Pipelock** (broader and more mature; Belay is far simpler and one small YAML file) and **Cupcake** on the hook side. Belay and a kernel sandbox like Leash are complementary, not substitutes.

---

## Limitations

Read these before you trust it.

- **Not a sandbox.** Belay sees tool **names and arguments**, not what a program does once it runs. An agent that can run arbitrary shell can obfuscate a command (`eval "$(echo … | base64 -d)"`) or build it at runtime, and Belay will not see through it. Pair Belay with the Claude Code sandbox, a container, or a kernel-level tool like Leash for a real boundary. The corpus deliberately contains no such evasion cases, because measuring the pack against attacks it can't see wouldn't be honest.
- **The corpus is self-built.** The rules and the test cases were written by the same person, so 71/71 measures coverage of anticipated patterns, not generalisation. The replay over 117k real calls is the more honest signal.
- **Hooks can be bypassed by the agent itself.** A PreToolUse hook is a config entry; an agent with shell access could edit `.claude/settings.json` to remove it. Belay mitigates this — the `coding-agent` pack denies writes to `.claude/settings.json`, `belay.yaml` and `.belay/`, and the installer sets `onFailure: "block"` (Claude Code ≥ 2.1.295) so a *crashing* hook blocks the call rather than failing open — but a determined agent with shell is outside Belay's threat model. Managed settings (admin-controlled) are the real answer there.
- **A Belay "allow" can't loosen Claude Code's own rules.** Belay never returns `allow` to Claude Code (that would skip its permission prompt); it only denies or asks. A settings `deny` rule still applies on top.
- **No content inspection.** Belay does not do DLP on tool *output*, PII masking, prompt-injection scanning of text, or tool-description scanning. It redacts secrets from its own audit log; it does not scan data flowing through tools.
- **Timing is honest but machine-dependent.** The hook's cost is dominated by Python interpreter start (~16 ms). Numbers above were measured under real background load; yours will differ.

---

## Tests & quality

```bash
pip install -e ".[dev]"
pytest -m "not integration"     # 112 fast tests, all offline
pytest -m integration           # 2 tests against the real filesystem MCP server (needs npx)
ruff check . && mypy src/agentbelay
```

114 tests cover policy compile and every rule type, the shell analyser, redaction, the Claude Code hook I/O against the [documented schema](https://code.claude.com/docs/en/hooks), the MCP proxy (fake server + the real one), the CLI, trace replay, the viewer, and the red-team corpus. CI runs them on Python 3.10–3.13, plus an integration job and a wheel-build check. Typed, `ruff`-clean, `mypy`-clean.

## Roadmap

The open-core path: a team dashboard over the JSONL audit logs, a central policy service with signed policy distribution, SSO and SIEM export, and tamper-evident (hash-chained) audit. The engine stays MIT.

## License

MIT. Built by [Sai Rakshith Muda](https://github.com/rakshithmuda22). Built on [agentspec](https://github.com/rakshithmuda22/agentspec).
