Metadata-Version: 2.5
Name: perfdigest-mcp
Version: 1.3.0
Summary: Local MCP server that digests development-loop reports — profilers (NVIDIA ncu, AMD rocprof, Linux perf, Apple Metal, chrome-trace), build tools (ptxas, clang -ftime-trace, cmake, ninja, cargo), benchmarks (criterion), CI run logs, and git change state — into token-efficient JSON for LLM coding agents.
Project-URL: Repository, https://github.com/onlyxItachi/PerfDigest-MCP
Project-URL: Issues, https://github.com/onlyxItachi/PerfDigest-MCP/issues
Author: Hamza Usta
License-Expression: Apache-2.0
License-File: LICENSE
License-File: NOTICE
Keywords: claude,codex,cuda,llm,mcp,metal,nsight,perf,profiler,rocm
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: Free Threading
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: System :: Benchmark
Requires-Python: >=3.10
Requires-Dist: mcp[cli]<3,>=2.3
Provides-Extra: all
Requires-Dist: ncu-report; (sys_platform != 'darwin') and extra == 'all'
Provides-Extra: amd
Provides-Extra: cpu
Provides-Extra: cuda
Requires-Dist: ncu-report; (sys_platform != 'darwin') and extra == 'cuda'
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == 'dev'
Provides-Extra: metal
Provides-Extra: nvidia
Requires-Dist: ncu-report; (sys_platform != 'darwin') and extra == 'nvidia'
Description-Content-Type: text/markdown

# perfdigest

Local **MCP server** that makes development-loop reports **token-efficient** for
LLM coding agents — profiler output first, and since v1.2 the rest of the loop's
feedback artifacts too (build traces, build diagnostics, benchmark results, CI
run logs, repo change state). It sits between the tool and the agent: reads the
report from disk and returns a small, structured, numeric digest the agent can
act on, while keeping a pointer back to the raw report for lazy expansion.

It is a **translator/router, not a judge** — interpretation ("is this kernel
memory-bound?") is the model's job. perfdigest only provides efficient,
deterministic access to clean numeric metrics across vendors and languages.

## Two operations — keep them separate

perfdigest deliberately splits what an agent does with a profiler into two tiers:

1. **Digest (read a report) — universal, available on every machine.** A report's
   origin is irrelevant to digesting it. An NVIDIA `.ncu-rep` (or its CSV export)
   captured on a CI/remote GPU runner can be pulled to a **Mac** and digested
   there — that is how you do CUDA work on a Mac via CI. Tier-1 is never gated by
   local hardware, only by whether a reader's dependency imports.
2. **Capture (produce a report) — platform-verified.** You cannot run `ncu` on a
   Mac, Metal on Linux, or hardware PMU counters under WSL2. `platform_capabilities`
   and `suggest_profile_command` gate this so the agent never spends context on a
   capture that can't run here, and redirect it to "capture elsewhere, digest here."

## The four feedback channels (v1.2 — the Development Observatory)

Every change a session makes propagates through the same four stages, each with
its own verbose artifact: *what changed* (RepoState) → *does it build, at what
cost* (BuildDigest) → *does it pass* (CIDigest) → *how fast is it* (PerfDigest).
perfdigest digests all four through the **same seven tools** — a backend is a
registry row, not a new API surface. One principle binds the whole matrix: we
**bind to artifacts, never to live APIs** — every reader parses a file on disk
whose format is frozen by its ecosystem's backward-compat pressure. (CI
*artifacts* need no dedicated backend at all: a downloaded `.ncu-rep` from a CI
runner is digested by `nsight` — capture anywhere, digest anywhere, by
composition.)

| Channel | Backend | `format` | Domain | Capture tool | Digest anywhere |
|---|---|---|---|---|---|
| Perf | `nsight` | `ncu-rep` | gpu_kernel | NVIDIA `ncu` (Linux/Windows) | needs `ncu-report` wheel |
| Perf | `nsight_csv` | `ncu-csv` | gpu_kernel | (export of `ncu`) | ✅ pure Python |
| Perf | `rocm` | `rocprof-csv` | gpu_kernel | AMD `rocprof` (Linux/Windows) | ✅ pure Python |
| Perf | `linux_perf` | `perf-stat-json`, `perf-report` | cpu_function | Linux `perf` | ✅ pure Python |
| Perf | `metal` | `metal-trace` | gpu_pass | Apple `xctrace` (macOS) | ✅ pure Python |
| Perf | `chrome_trace` | `torch-trace`, `chrome-trace` | framework_op + gpu_kernel | `torch.profiler` → `export_chrome_trace()` | ✅ pure Python |
| Build | `ptxas` | `ptxas-verbose` | kernel_codegen | `nvcc -Xptxas -v` (**no GPU needed**) | ✅ pure Python |
| Build | `clang_time_trace` | `clang-time-trace`, `ftime-trace` | build_phase | `clang -ftime-trace` | ✅ pure Python |
| Build | `cmake_profile` | `cmake-profile`, `cmake-trace` | build_phase | `cmake --profiling-format=google-trace` (≥3.18) | ✅ pure Python |
| Build | `ninja_log` | `ninja-log`, `ninja_log` | build_step | `.ninja_log` (byproduct of any `ninja` build) | ✅ pure Python |
| Build | `cargo_diag` | `cargo-diag`, `cargo-json` | build_diag | `cargo build --message-format=json` | ✅ pure Python |
| Build | `criterion` | `criterion`, `criterion-json` | benchmark | `cargo bench` (`target/criterion` dir) | ✅ pure Python |
| CI | `gha_log` | `gha-log`, `gh-run-log` | ci_step | saved `gh run view <id> --log` | ✅ pure Python |
| RepoState | `git_numstat` | `git-numstat`, `numstat` | repo_change | saved `git diff --numstat -M` | ✅ pure Python |

GPU backends share one vocabulary (`compute_pct_peak`, `dram_pct_peak`,
`l2_hit_rate`, `achieved_occupancy`, …); the CPU backend introduces a CPU
vocabulary (`ipc`, `cache_miss_rate`, `llc_miss_rate`, `branch_mispredict_rate`,
`self_pct`, …). The `ptxas` backend adds the **codegen layer**: runtime counters
say *what* is slow, `registers_per_thread` / `spill_stores_bytes` /
`static_smem_bytes` say *why the code became what it is* — captured at compile
time, so it works in any CI job with no GPU attached.

RepoState is deliberately **session-retrospective, not GitHub-shaped**: a saved
`numstat` snapshot is the token-cheap answer to "what has this session changed
so far", and it doubles as the provenance anchor that pins perf/build/CI reports
to a code state. It emits facts (files, line counts, honest `None` for binary
files) — never verdicts like "release ready"; concluding is the agent's job.
Future: Go, Java, and other perf-critical runtimes.

## Two load-bearing invariants (read before running)

1. **Suppress profiler stdout — write to a file.** e.g. `ncu --set full -o report.ncu-rep ./app`.
   If the profiler prints its summary to stdout, that raw table enters the agent's
   context *before* perfdigest runs — defeating the entire purpose.
2. **`None` means "not measured in this export", NEVER zero.** A metric the export
   does not contain is returned as `not_available_in_this_export`, not a fake `0.0`.
   A genuine `0.0` (e.g. zero branch divergence) is preserved. Silently returning
   zero = lying to the model = the worst bug this tool can have.

## Tools

Tier 1 — digest (any backend, any host):

- `summarize_report(report_ref, format, top_n=5)` → the N hottest units + core
  metrics in one call (ranked by `duration_us`, falling back to `self_pct`, then
  file order; reports duration coverage of the returned units)
- `list_kernels(report_ref, format)` → `[{name, index, duration_us, domain}]`
- `get_metrics(report_ref, format, kernel, metrics=None)` → compact digest
  (`metrics=None` → the backend's default core set)
- `compare_metrics(report_a, report_b, format, kernel, kernel_b=None)` → the
  measure→edit→measure tool: `{a, b, delta, delta_pct}` per metric with
  `delta = b − a`; a metric missing on either side yields an honest
  `not_available_in_this_export` delta, never a fake `0.0`
- `expand(report_ref, format, kernel, section)` → raw vendor metrics (the safety valve)

Reports are parsed once per file version (an mtime/size-keyed cache), so
repeated digests of the same report cost no re-parse.

Tier 2 — capture advisory (platform-verified):

- `platform_capabilities()` → machine identity + `can_digest` (universal) vs
  `can_capture_here` (gated)
- `suggest_profile_command(backend, target)` → the correct, platform-aware
  invocation, or a refusal that redirects to the tier-1 path

`format` is **mandatory** — the agent passes what it produced; a path says *where*
a file is, not *what* format it is.

## Install & connect

```bash
uvx perfdigest-mcp                      # run from PyPI (downloadable)
uv tool install "perfdigest-mcp[nvidia]"  # + NVIDIA native binary reader (Linux/Windows)
```

> PyPI/install name is **`perfdigest-mcp`**; the command and import package are
> `perfdigest` (e.g. `uvx perfdigest-mcp`, `import perfdigest`).

Claude Code and OpenAI Codex setup (both stdio MCP): see [`docs/clients.md`](docs/clients.md).

Version 1.3.0 uses [MCP Python SDK 2](docs/mcp-sdk-v2-migration.md) and serves
both legacy MCP clients and protocol 2026-07-28 over the same stdio command,
while preserving the seven tools and their v1.2.1 report contracts.

## Status

**v1.3.0** — MCP SDK 2 with legacy-client compatibility

The Python server now uses `MCPServer` and `mcp[cli]>=2.3,<3` (locked at 2.3.0).
SDK 1 clients continue to initialize with protocol 2025-11-25. Modern clients can
use `server/discover` or send 2026-07-28 requests without an initialization
handshake. The seven tool schemas, mandatory format, missingness and raw evidence
provenance remain covered against the actual published v1.2.1 package.

Validation covers 229 tests plus 16 native-report fixture skips, real stdio
clients and clean-wheel installs across Python 3.10–3.14/3.14t on
Linux/macOS/Windows where dependencies install. **Windows 3.14t is NOT EXECUTED**:
pywin32 312 has no applicable distribution. Only that exact installation gap is
classified; every other installation or test failure is strict. No HTTP service,
hardware capture or new performance claim is introduced. See the
[release notes](docs/releases/v1.3.0.md) and
[compatibility report](docs/mcp-sdk-v2-migration.md).

**v1.2.1** — MCP SDK v1 compatibility hotfix

This legacy version requires `mcp[cli]>=1.2,<2`, preserving the FastMCP interface
used by this release when MCP SDK 2 is available. The seven tool names and
parameters and the Python 3.10+ support range are unchanged.
This release also packages fixes already merged after v1.2.0 in #32 and #33:
torch.compile envelopes carry `[dispatch]` tags; summaries expose distinct-name
coverage and use `math.fsum` for stable totals; comparisons add `domain_a` and
`domain_b` while retaining the existing `domain` alias. Usage guidance explains
fusion/group comparisons, Ninja time origins and names omitted by top-N.
Its release validation passed 216 tests with 16 real-report fixture skips.
Windows 3.14t tests did not execute: the `pywin32==312` install failure was the
sole allowed CI failure in that legacy release.
Clean wheel installs and actual stdio calls to all seven tools were checked on
Python 3.10 and 3.14 with MCP 1.30.0, as well as the locked MCP 1.27.2 environment.
The declared MCP 1.2 floor is retained: forcing that old SDK with current
Pydantic fails on a removed private import, while its compatible Pydantic 2.10.6
and pydantic-settings 2.7.1 combination passes. Normal fresh installs select a
recent SDK 1 release.

**v1.2.0** — the Development Observatory release

Seven new backends extended the digest matrix from profilers to the whole
edit→build→verify→measure loop. BuildDigest: `clang_time_trace` (compiler-phase
traces, with `[total]` aggregate tagging), `cmake_profile` (configure-step
traces — cmake emits bare `B`/`E` event pairs, stack-folded honestly),
`ninja_log` (per-edge build timings, v5–v7 logs), `cargo_diag` (rustc
diagnostics — a backend whose durations are *honestly absent*), `criterion`
(Rust benchmark estimates; first directory-shaped report ref). CIDigest:
`gha_log` (saved `gh run view --log` files, error/warning annotations,
previous-green comparison proven on real two-run fixtures). RepoState:
`git_numstat` (session-retrospective change digest; binary files surface as
honest absence, never `0.0`). Zero new tools — every backend rides the same
seven; 206 tests. Every fixture in the release was produced by the real tool it
mimics — several briefs were corrected by what the artifacts actually contained.

**v1.1.1** — contract-hardening hotfix (issues #2/#4/#5): registry format
collisions are detected on the normalized key (a case-variant format can no
longer silently hijack a backend); `metrics=[]` now means an empty request
(only `None` selects the default core set); exact unit names win over numeric
index interpretation, with `#3` as the explicit index form; unreadable backends
surface a diagnostic in `platform_capabilities` (`can_digest_issues`) instead
of a silent `False`; and publishing is gated — a tag only reaches PyPI after
the full suite, a clean-venv wheel install, and a tool-surface check pass.

**v1.1.0** — the agent-loop release: `compare_metrics` (before/after deltas),
`summarize_report` (one-call top-N), a parse-once report cache, the `ptxas`
codegen backend (compile-time registers/spills/smem; capture needs `nvcc`, not a
GPU), and the `chrome_trace` backend (torch/Kineto and any Chrome-trace emitter:
framework ops + the CUDA kernels they launched, one report). Also normalizes
`perf` scope-modifier event names (`cycles:u` on `perf_event_paranoid>=2`
hosts).

**v1.0.0** — multi-backend registry; NVIDIA (native + CSV), AMD HIP, Linux perf
(C++/Rust), and Apple Metal adapters; platform capability gating; cross-client
config. Validated on the Linux/macOS/Windows CI matrix (pure-Python readers run
hardware-free against committed fixtures). Real binary-capture tests are
fixture-gated and skip without the device.

## Measured behavior

Every number here comes from a real capture on real hardware, published with the
workload that produced it in
[PerfDigest-MCP-Bench](https://github.com/onlyxItachi/PerfDigest-MCP-Bench)
(local [`eval/README.md`](eval/README.md) is the pointer). Ratios are digest
payload vs the raw report an agent would otherwise read.

| Study | Subject | What it measured |
|---|---|---|
| Demanding workloads **pt. 1** | closed-source cuBLAS GEMMs; torch.compile inference | four vendor kernel families diagnosed from counters alone — no source to read — at **29–46×**; Inductor fusion visible structurally at **74–115×** |
| **pt. 2** | a full `torch.compile` **training** step (forward + backward + AdamW) | Inductor-fused *backward* kernels diagnosed memory-bound at **96–450×**; and a `compare_metrics` pairing caught reporting the **wrong sign** of a fusion win when one kernel is compared against what is really its multi-kernel replacement |
| **pt. 3** | one C++ project through **every** build lens (`cmake_profile` + `ninja_log` + `clang_time_trace`) | a template-heavy TU dominating the build 5× over the next edge, its cost split by `-ftime-trace` into a single recursive-`constexpr` evaluation (37% of compile time) and STL instantiation pressure (**10.7×** nesting overlap, which is why aggregate spans are tagged); **261–5,440×** on the trace lenses |
| **pt. 4** | the CUDA optimization loop (naive → coalesced → vectorized) | v0→v1 a real **2.64×** win explained by `dram_pct_peak` 36%→90%; v1→v2 an honest **non-win** — a genuine −20.9% instruction cut bought nothing once already at ~90% of DRAM peak; `ptxas` foreshadows the ranking only *below* the roofline ceiling; **11–86×** |
| Interpreter matrix | six CPythons, 3.10 → free-threaded 3.14t | digests **byte-identical** across all six for 10 of 11 pure-Python backends (the one divergence traced to CPython 3.12's `sum()` change); readers parse in < 2 ms; the parse-once cache holds correctness under 8-thread free-threaded contention at **2.36×** GIL throughput |

The low ratios are published next to the high ones on purpose. A `.ninja_log` digests
at **0.80×** and its `summarize_report` at **0.49×** — a build log is already terse,
so there is nothing to compress; what the digest buys there is a *uniform contract*,
not fewer tokens. A measurement tool that only reported its wins would not be one.

## Related / similar projects

perfdigest was authored independently; these adjacent MCP servers occupy a nearby
space and likely work well for their narrower scope. The differences are the
reason perfdigest exists:

- **nsys-mcp** (NVIDIA Nsight *Systems*) — profiles binaries and aggregates trace
  timeline stats. perfdigest targets Nsight *Compute* per-kernel counters, is
  read-only (does not run the profiler), and spans multiple vendors.
- **pprof-analyzer-mcp** / **Profiler-MCP** — Go (and Go/Python/Java) CPU/memory
  profiles, often rendering flamegraphs. perfdigest is a numeric digester focused
  on token/context efficiency rather than visualization.

What is distinct here: the **multi-backend matrix** (NVIDIA + AMD HIP + CPU perf +
Metal under one neutral contract), the **token-efficiency thesis** with the
`None`≠`0.0` honesty rule, and the **read/capture split with platform capability
gating** (digest anywhere, capture only where supported).

## License

Apache-2.0 — see [`LICENSE`](LICENSE) and [`NOTICE`](NOTICE).
