Metadata-Version: 2.4
Name: neuronscope-cli
Version: 0.1.2
Summary: CLI and MCP server for tracing which neurons and attention heads drive a language model's output, built on TransformerLens.
Project-URL: Homepage, https://github.com/RudrenduPaul/NeuronScope
Project-URL: Repository, https://github.com/RudrenduPaul/NeuronScope
Project-URL: Issues, https://github.com/RudrenduPaul/NeuronScope/issues
Project-URL: Author - Rudrendu Paul, https://github.com/RudrenduPaul
Project-URL: Author - Sourav Nandy, https://github.com/Sourav-Nandy-ai
Author: Rudrendu Paul, Sourav Nandy
License: MIT
License-File: LICENSE
Keywords: cli,interpretability,llm,mcp,mechanistic-interpretability,transformer-lens
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: click>=8.1
Requires-Dist: mcp>=1.0
Requires-Dist: pydantic>=2.0
Requires-Dist: rich>=13.0
Requires-Dist: torch>=2.0
Requires-Dist: transformer-lens>=2.0
Provides-Extra: dev
Requires-Dist: pytest-cov>=4.0; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Description-Content-Type: text/markdown

# NeuronScope

Ask a language model "why did you say that" and get back the actual attention heads and
neurons responsible, as JSON, from the command line or from an agent over MCP.

[![CI](https://github.com/RudrenduPaul/NeuronScope/actions/workflows/ci.yml/badge.svg)](https://github.com/RudrenduPaul/NeuronScope/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/neuronscope-cli.svg)](https://pypi.org/project/neuronscope-cli/)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/RudrenduPaul/NeuronScope/blob/main/LICENSE)

![NeuronScope tracing a real gpt2 prediction from the command line, showing the top attention heads and MLP neurons responsible for the output](https://raw.githubusercontent.com/RudrenduPaul/NeuronScope/main/docs/demo.gif)

## Install

```bash
pip install neuronscope-cli
```

That gets you the `neuronscope` command. To install from source instead (for development or to
track `main`):

```bash
git clone https://github.com/RudrenduPaul/NeuronScope
cd NeuronScope
pip install -e .
```

**First run of any command** downloads the requested model from the HuggingFace Hub (`gpt2`
is about 500MB) and prints two lines to stderr that are expected, not errors: a CPU-fallback
notice if you don't have a CUDA GPU, and an unauthenticated-HF-Hub rate-limit notice. Neither
one means anything broke.

## What it does

NeuronScope is a CLI and [MCP](https://modelcontextprotocol.io) server built on top of
[TransformerLens](https://github.com/TransformerLensOrg/TransformerLens). TransformerLens
does the actual model loading, hooking, and activation math; NeuronScope adds a stable CLI,
a versioned JSON schema, and an MCP server around it, so a script or an agent can ask "which
components drove this output" without writing TransformerLens code directly.

- **`trace`**: runs a prompt through the model and ranks attention heads by direct logit
  attribution to the predicted token, and MLP neurons by activation magnitude at the final
  prompt position.
- **`activations`**: dumps shape, mean, std, min/max, and the max-activating sequence
  position for every layer's residual stream, MLP neuron activations, and attention pattern.
- **`patch`**: zero-ablates one component (`resid_pre`, `resid_mid`, `resid_post`,
  `attn_out`, `mlp_out`, or `mlp_post`) at a given layer and reports how the predicted token
  and its logit changed.
- **`circuit`**: a best-effort automated circuit sketch. Ranks candidate heads/neurons by
  logit attribution, then measures each one's individual causal effect via single-component
  ablation. This is not full path-patching with clean/corrupted prompt pairs and does not
  capture interaction effects between components. The `--json` output says so explicitly in
  its `method` field.
- Every command supports `--json` for a `schema_version`-stamped document instead of a
  table, and the same four operations are exposed as MCP tools returning the identical
  shape via `.model_dump()`, so a CLI call and an MCP tool call produce the same document
  for the same input.
- Model support is whatever `transformer_lens.HookedTransformer.from_pretrained` supports.
  As of the `transformer-lens` version this repo currently depends on, that's 247 pretrained
  checkpoints and aliases (`OFFICIAL_MODEL_NAMES`), covering GPT-2, Pythia, Llama, Gemma,
  Qwen, and more. Small models like `gpt2` run comfortably on CPU.

NeuronScope does not replace TransformerLens, [nnsight](https://nnsight.net/),
[SAELens](https://github.com/jbloomAus/SAELens), Anthropic's
[circuit-tracer](https://github.com/decoderesearch/circuit-tracer), or
[Neuronpedia](https://www.neuronpedia.org/). It wraps TransformerLens for one narrower job:
fast, scriptable, agent-callable component tracing on a single prompt. It leaves deeper
mechanistic work (SAE training, transcoder-based circuit graphs, hosted feature browsing) to
those tools.

## Quickstart

```bash
neuronscope trace gpt2 "The capital of France is Paris. The capital of Japan is" --top-k 5
```

Real output from this exact command (stderr trimmed to the two expected warnings mentioned
above):

```
Prompt: The capital of France is Paris. The capital of Japan is
Predicted next token: ' Tokyo'
Top attention heads (by direct logit
            attribution)
┏━━━━━━━┳━━━━━━┳━━━━━━━━━━━━━━━━━━━┓
┃ Layer ┃ Head ┃ Logit attribution ┃
┡━━━━━━━╇━━━━━━╇━━━━━━━━━━━━━━━━━━━┩
│     9 │    8 │            4.0679 │
│     8 │   11 │            2.9028 │
│    10 │    7 │           -1.4782 │
│     8 │   10 │           -1.3999 │
│    10 │    0 │            1.1424 │
└───────┴──────┴───────────────────┘
Top MLP neurons (by activation
          magnitude)
┏━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┓
┃ Layer ┃ Neuron ┃ Activation ┃
┡━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━┩
│    10 │     97 │     7.8394 │
│    11 │    611 │     4.6954 │
│    11 │   2997 │     4.6468 │
│    10 │   1793 │     4.5443 │
│     9 │   1460 │     4.4196 │
└───────┴────────┴────────────┘
```

gpt2 predicts `Tokyo` correctly, and head `L9H8` is the single biggest contributor to that
prediction. Add `--json` to get the machine-readable version of the same result:

```bash
neuronscope trace gpt2 "The capital of France is Paris. The capital of Japan is" --top-k 3 --json
```

```json
{
  "schema_version": 1,
  "operation": "trace",
  "model": {
    "requested_name": "gpt2",
    "resolved_name": "gpt2",
    "backend": "transformer_lens",
    "device": "cpu",
    "n_layers": 12,
    "n_heads": 12,
    "d_model": 768,
    "d_mlp": 3072
  },
  "prompt": "The capital of France is Paris. The capital of Japan is",
  "predicted_token": " Tokyo",
  "predicted_token_id": 11790,
  "top_neurons": [
    { "layer": 10, "neuron_index": 97, "activation": 7.839381217956543 },
    { "layer": 11, "neuron_index": 611, "activation": 4.695372581481934 },
    { "layer": 11, "neuron_index": 2997, "activation": 4.646785736083984 }
  ],
  "top_heads": [
    { "layer": 9, "head_index": 8, "logit_attribution": 4.067923545837402 },
    { "layer": 8, "head_index": 11, "logit_attribution": 2.9028172492980957 },
    { "layer": 10, "head_index": 7, "logit_attribution": -1.4781968593597412 }
  ]
}
```

## CLI reference

Every command takes `MODEL` (any name `HookedTransformer.from_pretrained` accepts, for
example `gpt2` or `EleutherAI/pythia-70m`) and `PROMPT` as positional arguments.

| Command | Extra flags | What it does |
|---|---|---|
| `neuronscope trace MODEL PROMPT` | `--top-k INTEGER` (default 10), `--json` | Ranks top attention heads (logit attribution) and MLP neurons (activation magnitude) for the predicted next token |
| `neuronscope activations MODEL PROMPT` | `--json` | Dumps per-layer activation summary stats (residual stream, MLP, attention pattern) |
| `neuronscope patch MODEL PROMPT` | `--layer INTEGER` (required), `--component [resid_pre\|resid_mid\|resid_post\|attn_out\|mlp_out\|mlp_post]` (required), `--json` | Zero-ablates one component and reports the logit/prediction delta |
| `neuronscope circuit MODEL PROMPT` | `--top-k INTEGER` (default 10), `--json` | Best-effort circuit sketch via ranked single-component ablation |
| `neuronscope mcp-server` | none | Starts the MCP server over stdio |

Global: `neuronscope --version`, `neuronscope <command> --help`. Exit codes: `0` success,
`1` a runtime error (prompt too long for the model's context window, `--layer` out of range,
etc.), `2` a Click usage error (bad flags), `3` an unsupported model name.

## MCP server

```bash
neuronscope mcp-server
```

Starts an MCP server over stdio that exposes `trace`, `activations`, `patch`, and `circuit`
as MCP tools, with the same arguments and the same JSON schema as the CLI's `--json` output.

To register it with an MCP host, add:

```json
{
  "mcpServers": {
    "neuronscope": {
      "command": "neuronscope",
      "args": ["mcp-server"]
    }
  }
}
```

- **Claude Code** reads this from a project-level `.mcp.json` in your repo root, or you can
  add it with `claude mcp add neuronscope -- neuronscope mcp-server`.
- **Claude Desktop** reads this from its `claude_desktop_config.json`
  (`~/Library/Application Support/Claude/claude_desktop_config.json` on macOS,
  `%APPDATA%\Claude\claude_desktop_config.json` on Windows), under the same
  `"mcpServers"` key.

## How it compares

All five of these are real, actively maintained projects doing different jobs. This table
compares CLI/JSON-agent-output surface and model coverage, not depth of interpretability
research, where TransformerLens, nnsight, SAELens, circuit-tracer, and Neuronpedia are all
more mature than NeuronScope. Star counts, release info, and last-push dates below were
pulled from each project's GitHub API on 2026-07-19 and will drift over time; check the repos
directly for current numbers.

| Project | Stars | Last activity | CLI | Agent-callable structured output | Model coverage |
|---|---|---|---|---|---|
| [TransformerLens](https://github.com/TransformerLensOrg/TransformerLens) | 3,689 | v3.5.1 released 2026-07-01, pushed 2026-07-16 | No (Python library) | No | 247 pretrained checkpoints/aliases (its own official list) |
| [nnsight](https://github.com/ndif-team/nnsight) | 995 | v0.7.0 released 2026-05-05, pushed 2026-07-14 | No (Python library) | No (returns tensors/Python objects) | Any HuggingFace or PyTorch model generically, no fixed list |
| [circuit-tracer](https://github.com/decoderesearch/circuit-tracer) (Anthropic-authored, moved from `safety-research/circuit-tracer`) | 2,864 | v0.5.2 released 2026-07-18 | Yes | JSON attribution-graph export; no MCP server | Fixed transcoder allowlist: Gemma-2 (2B), Gemma-3 (270M-27B), Llama-3.2 (1B), Llama-3.1 (8B Instruct), Qwen-3 (0.6B-14B), GPT-OSS (20B) |
| [SAELens](https://github.com/jbloomAus/SAELens) | 1,476 | v6.46.0 released 2026-07-13, pushed 2026-07-13 | No (Python library) | No | Any PyTorch model generically; deepest integration is with TransformerLens |
| [Neuronpedia](https://github.com/hijohnnylin/neuronpedia) | 1,070 | continuously deployed, tag v1.0.795 on 2026-07-17 | No (hosted web app + REST API) | REST API returns JSON; MCP access exists only via an unofficial third-party wrapper, not the official repo | Models loadable through TransformerLens's model table (GPT-2, Gemma-2, Llama, DeepSeek, etc.) |
| **NeuronScope** (this project) | New, pre-release | this commit | Yes | Yes: `--json` on every command, plus a native MCP server returning the same schema | Whatever TransformerLens's `HookedTransformer.from_pretrained` supports: 247 checkpoints/aliases |

The honest differentiation is narrow: NeuronScope is the only one of these with a CLI, a
native MCP server, and a versioned JSON schema together in one package, and it's
model-agnostic across whatever TransformerLens supports rather than pinned to a fixed
transcoder allowlist like circuit-tracer. It is not more capable, more mature, or more
widely used than any of these projects.

## What is NeuronScope and why does it exist

TransformerLens gives you a Python API for loading a model and running hooked forward
passes. That's the right interface for a research notebook. It's the wrong interface for a
script that needs a subprocess call and a JSON document back, or for an agent that needs a
tool it can call over MCP. NeuronScope exists to be that second interface: the same
underlying computation, wrapped so a CLI invocation or an MCP tool call gets back a
schema-versioned document instead of a Python object graph.

## FAQ

**Is this a replacement for TransformerLens, nnsight, SAELens, circuit-tracer, or
Neuronpedia?**
No. NeuronScope is built directly on TransformerLens and does not do anything TransformerLens
itself can't already do at a lower level. It doesn't train SAEs (SAELens), do full
path-patching circuit discovery with transcoders (circuit-tracer), give you a Python-native
tracing context manager for arbitrary PyTorch models (nnsight), or host a browsable feature
database (Neuronpedia). It's a CLI and MCP wrapper around one slice of TransformerLens's
functionality.

**What models are supported?**
Anything `transformer_lens.HookedTransformer.from_pretrained` supports, which today is 247
checkpoints and aliases spanning GPT-2, Pythia, Llama, Gemma, Qwen, and others. Run
`python -c "from transformer_lens.loading_from_pretrained import OFFICIAL_MODEL_NAMES; print(len(OFFICIAL_MODEL_NAMES))"`
in your own environment to get the exact count for your installed version, since
TransformerLens adds models over time.

**Does it need a GPU?**
No. Small models like `gpt2` run fine on CPU; that's what the test suite and the quickstart
above run on. Larger models will be slow on CPU. NeuronScope does not auto-select Apple
Silicon's MPS backend even when available, because PyTorch's MPS backend can silently
produce incorrect values for some ops that this project's activation-patching math depends
on being exact. Pass `device="mps"` explicitly in your own code if you want it anyway.

**Is it safe to expose the MCP server to an untrusted agent?**
Only with resource limits in place. See Known limitations below.

## Known limitations

- **`circuit` is an approximation.** It ranks components by logit attribution and measures
  each one's individual causal effect via single-component zero-ablation on one prompt. It
  does not do full path-patching with clean/corrupted prompt pairs, and it will not catch
  interaction effects between components. The `--json` output states this in its `method`
  field so a caller doesn't have to trust prose to know the caveat.
- **No size cap or timeout on model loading or forward passes.** NeuronScope loads whatever
  model weights the caller asks for and runs the forward pass to completion, with no built-in
  limit on model size or wall-clock time. If you run the MCP server somewhere an untrusted
  agent can call it, put a resource limit around the process (a cgroup, `ulimit`, or a
  container memory/CPU cap) rather than relying on NeuronScope to refuse an oversized
  request on its own.

## Contributing

Issues and pull requests are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md) for dev setup,
where the code lives, and what a PR needs before it merges. Quick version:

```bash
pip install -e ".[dev]"
pytest -v
```

CI runs the same suite on Python 3.10, 3.11, and 3.12 on every push and pull request against
`main`. The suite covers 87% of `neuronscope/` (`pytest --cov=neuronscope`), with the MCP
server's less-exercised paths (specific error branches) the main gap.

## License

MIT. See [LICENSE](LICENSE).
