Metadata-Version: 2.4
Name: kibitzer
Version: 0.9.0
Summary: Claude Code extension that watches agent tool calls and suggests structured alternatives
Project-URL: Homepage, https://github.com/teague/kibitzer
Project-URL: Repository, https://github.com/teague/kibitzer
Project-URL: Issues, https://github.com/teague/kibitzer/issues
Project-URL: Documentation, https://kibitzer.readthedocs.io
Author: Teague Sterling
License-Expression: MIT
License-File: LICENSE
Keywords: agent,claude-code,developer-tools,hooks,mcp,toolcraft
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.10
Requires-Dist: click>=8.1
Requires-Dist: lackpy-lang>=0.12
Requires-Dist: mcp[cli]>=1.0
Provides-Extra: dev
Requires-Dist: duckdb>=1.5; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.21; extra == 'dev'
Requires-Dist: pytest-cov>=4.0; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Requires-Dist: ruff>=0.1; extra == 'dev'
Provides-Extra: docs
Requires-Dist: mkdocs-material>=9.0; extra == 'docs'
Requires-Dist: mkdocs>=1.5; extra == 'docs'
Provides-Extra: fledgling
Requires-Dist: fledgling-mcp>=0.12; extra == 'fledgling'
Provides-Extra: pluckit
Requires-Dist: ast-pluckit>=0.13; extra == 'pluckit'
Description-Content-Type: text/markdown

# Kibitzer

*The person watching your chess game who can't help offering opinions.*

Kibitzer is a [Claude Code](https://claude.ai/code) extension that watches how agents use tools and suggests better alternatives. It applies mode-based path protection to write tools (canonicalized, fail-closed) and to the Bash write vectors it can see statically, intercepts bash commands that have structured alternatives, and coaches agents toward more effective tool usage — all without an LLM in the decision loop. It is a guidance layer that composes *under* the Claude Code permission system and an OS-level sandbox, not a replacement for either (see [Scope and limits](#scope-and-limits)).

[![PyPI](https://img.shields.io/pypi/v/kibitzer)](https://pypi.org/project/kibitzer/)
[![Python](https://img.shields.io/pypi/pyversions/kibitzer)](https://pypi.org/project/kibitzer/)
[![License](https://img.shields.io/github/license/teague/kibitzer)](LICENSE)
[![Docs](https://img.shields.io/readthedocs/kibitzer)](https://kibitzer.readthedocs.io)

## Install

```bash
pip install kibitzer
cd your-project/
kibitzer init --hooks --mcp
```

This registers PreToolUse/PostToolUse hooks in `.claude/settings.json` and starts an MCP server with two tools the agent can call: `ChangeToolMode` and `GetFeedback`.

For richer coaching with [Fledgling](https://github.com/teague/source-sextant) conversation analytics:

```bash
pip install kibitzer[fledgling]
```

## What it does

### Path protection

Each mode defines which paths the agent can write to. The path guard checks every `Edit`, `Write`, and `NotebookEdit` call, and statically scans `Bash` commands for common write vectors (redirections, `tee`, `cp`, `mv`, `rm`, `sed -i`, `ln`, ...).

Before comparing, the guard **canonicalizes** the target against the project directory — symlinks are followed, `.`/`..` are collapsed, absolute and relative spellings of the same location compare equal — and matches by path segment, so `src/` never matches `src_secret/`. The guard **fails closed**: an unknown mode, a missing target path, an unparseable Bash command in a restricted mode, or an internal guard error denies the write.

```
Mode        Writable                  Use case
─────────── ───────────────────────── ───────────────────────────
free        everything                prototyping, floor only
implement   src/, lib/, tests/, test/ normal dev (tests included)
test        tests/, test/, spec/      writing tests — source protected
docs        docs/, README.md          documentation only
explore     nothing                   read-only investigation
review      nothing                   read-only code review
```

Writable entries may also be absolute or `~`-prefixed glob patterns, expanded at check time — the implement/test/docs defaults include `~/.claude/projects/*/memory/` so Claude auto-memory writes don't require a mode flip.

Independent of the mode, a **safety floor** deny-list runs first on every write (including statically extracted Bash write targets): secret dirs (`~/.ssh`, `~/.aws`, ...), system paths, raw `.git` internals, and writes outside any git working tree are blocked even in `free` mode. It's configurable via `[floor]` in `.kibitzer/config.toml` and fails open on its own internal errors.

When a write is denied, the agent sees why and how to fix it:

```
Path 'notebooks/explore.ipynb' is not writable in the current mode (writable: ['src/', 'lib/', 'tests/', 'test/', '~/.claude/projects/*/memory/']).
Use the ChangeToolMode tool to switch modes.
```

In testing, agents consistently read this message and call `ChangeToolMode` to switch — no documentation or pre-training needed.

#### Scope and limits

Path protection is one layer, not a sandbox:

- **Bash coverage is static.** The guard analyzes the command string; it is not a shell interpreter. Writes hidden behind interpreter one-liners (`python -c ...`), scripts fed on stdin, `make` targets, or git commands that mutate the tree are not visible to it. Opaque write constructs it *can* spot — `xargs rm`, process substitution targets, unparseable quoting — are denied in restricted modes rather than waved through.
- **Check-then-act window.** Symlinks are resolved when the call is checked; a race that swaps a path component between the check and the actual write is not preventable at this layer.
- **`free` mode skips the mode guard by design** (`writable = ["*"]`) and is the default; only the mode-independent safety floor still applies. Set a restricted `default_mode` in `.kibitzer/config.toml` if you want the full guard active from the first call.

For a hard boundary, compose kibitzer with the Claude Code permission system and an OS-level sandbox (e.g. an umwelt-compiled policy under nsjail). Kibitzer keeps a cooperating-but-fallible agent on task; the sandbox bounds what a misbehaving process can do.

### Interception

Interceptor plugins watch Bash calls for commands that have structured alternatives:

| Bash command | Suggested alternative | Plugin |
|---|---|---|
| `git add -A && git commit -m '...'` | `jetsam save` | jetsam |
| `pytest tests/` | `blq run test` | blq |
| `grep -rn 'def handler' src/` | `FindDefinitions(...)` | fledgling |

Three interception modes form a ratchet — start in `observe` (log silently), graduate to `suggest` (show alternative), then `redirect` (deny bash, require structured tool). Each graduation is a one-line config change.

### Coaching

The coach fires every N tool calls and detects patterns from ~250 experimental runs. Suggestions only reference tools the agent actually has — discovered from `.mcp.json` at runtime.

**State-based patterns (always available):**

- **Repeated edit failures** — "Edit failed 3 times on src/handler.py. Try Read() first to see exact content."
- **Edit streak without tests** — "You've made 7 edits without running tests." (mentions `blq run test` if blq is available)
- **Sequential file reads** — "You've read 5 files one at a time." (mentions `FindDefinitions` if fledgling is available)
- **Bash-heavy usage** — "You've run 6 bash commands without using structured tools."
- **Analysis loop** — "You've spent 18 turns reading without changes. Start with the most confident fix."
- **Semantic tool underuse** — "FindDefinitions shows all functions in one call." (only fires if fledgling is available)
- **Mode oscillation** — "Frequent mode switches. Consider using free mode."

**TDD patterns:**

- **Test overfit** — "test_auth.py has been edited 4 times. Stabilize test expectations before adjusting further."
- **Implement before test** — "You edited source before writing tests. Consider starting with a failing test."

**Fledgling-powered patterns (when fledgling is installed):**

- **Repeated search patterns** — "You've searched for 'def handle_request' 4 times via Grep."
- **Replaceable bash commands** — "You've run 'grep' 3 times. FindDefinitions provides structured output."

All patterns are mode-aware: the analysis loop doesn't fire in explore mode (not editing is correct there), edit-without-test doesn't fire in docs mode (docs don't need tests).

### Auto-transitions

The mode controller watches for failure patterns:
- 3+ consecutive failures → auto-switch to `explore`
- 20+ turns in explore → auto-switch back to `implement`

An oscillation guard prevents rapid switching: if the agent just left a mode (< 5 turns), it won't auto-switch back. After 6+ total switches, auto-transitions stop.

## Composes with Claude Code's permission model

The Claude Code harness has its own permission system (per-tool allow/deny, prompting before sensitive actions). Kibitzer's modes layer *on top of* these permissions — not beside them. The two regulate at different granularities:

- **Harness permissions**: coarse, universal, set at install — "can this agent ever call Bash?"
- **Kibitzer modes**: fine, contextual, changed per task — "is Bash appropriate given the current mode?"

They compose via set intersection: a tool call succeeds iff the harness permits it AND (if kibitzer is active) the current mode permits it. Kibitzer can narrow the harness's permissions; it can never widen them.

When a call is denied, the response identifies which layer denied it so the agent can respond correctly — switching modes (kibitzer) or asking the user to grant permission (harness) are different remedies.

## MCP tools

The agent can call two tools explicitly:

**`ChangeToolMode(mode, reason?)`** — Switch modes. Returns the new mode's writable paths and strategy.

**`GetFeedback(status?, suggestions?, intercepts?)`** — Check current status, get coaching suggestions, and see which bash commands have been intercepted.

## Configuration

Override defaults in `.kibitzer/config.toml`:

```toml
# Monorepo: widen writable paths
[modes.implement]
writable = ["packages/core/src/", "packages/api/src/"]

# Add custom modes
[modes.deploy]
writable = ["infra/", "deploy/"]
strategy = "Verify before applying."

# Graduate jetsam to suggest mode
[plugins.jetsam]
mode = "suggest"

# More aggressive coaching
[coach]
frequency = 3
```

## Python API

Use kibitzer from Python without the hook protocol:

```python
from kibitzer import KibitzerSession

with KibitzerSession(project_dir=".") as session:
    # Check if a tool call is allowed
    result = session.before_call("Edit", {"file_path": "src/auth.py"})

    # Record a completed tool call
    session.after_call("Edit", {"file_path": "src/auth.py"}, success=True)

    # Batch-validate a program's planned calls
    violations = session.validate_calls([
        {"tool": "Edit", "input": {"file_path": "tests/foo.py"}},
    ])
```

Full API docs at [kibitzer.readthedocs.io/python-api](https://kibitzer.readthedocs.io/python-api/).

## Coordinates with

Kibitzer suggests but never wraps these tools — each is independent:

- **[blq](https://github.com/teague/lq)** — structured build/test capture
- **[jetsam](https://github.com/teague/jetsam)** — git workflow acceleration
- **[Fledgling](https://github.com/teague/source-sextant)** — AST-aware code intelligence

None are required. Kibitzer degrades gracefully — path guard and coach work with nothing else installed. When tools are available, suggestions reference them specifically. When they're not, suggestions give generic advice.

## Documentation

Full docs at [kibitzer.readthedocs.io](https://kibitzer.readthedocs.io):

- [Modes](https://kibitzer.readthedocs.io/modes/) — path protection, switching, auto-transitions
- [Coach](https://kibitzer.readthedocs.io/coach/) — all patterns, experimental evidence, model dependency
- [Interceptors](https://kibitzer.readthedocs.io/interceptors/) — the observe/suggest/redirect ratchet
- [Configuration](https://kibitzer.readthedocs.io/configuration/) — full config.toml reference, resilience, optional deps
- [Architecture](https://kibitzer.readthedocs.io/architecture/) — how the pieces fit together
- [Integration](https://kibitzer.readthedocs.io/integration/) — blq, jetsam, fledgling, superpowers

## License

MIT
