perfdigest

Local MCP server that makes development-loop reports token-efficient for LLM coding agents — profiler output first, and since v1.2 the rest of the loop's feedback artifacts too (build traces, build diagnostics, benchmark results, CI run logs, repo change state). It sits between the tool and the agent: reads the report from disk and returns a small, structured, numeric digest the agent can act on, while keeping a pointer back to the raw report for lazy expansion.

It is a translator/router, not a judge — interpretation ("is this kernel memory-bound?") is the model's job. perfdigest only provides efficient, deterministic access to clean numeric metrics across vendors and languages.

Two operations — keep them separate

perfdigest deliberately splits what an agent does with a profiler into two tiers:

1. Digest (read a report) — universal, available on every machine. A report's origin is irrelevant to digesting it. An NVIDIA .ncu-rep (or its CSV export) captured on a CI/remote GPU runner can be pulled to a Mac and digested there — that is how you do CUDA work on a Mac via CI. Tier-1 is never gated by local hardware, only by whether a reader's dependency imports.
2. Capture (produce a report) — platform-verified. You cannot run ncu on a Mac, Metal on Linux, or hardware PMU counters under WSL2. platform_capabilities and suggest_profile_command gate this so the agent never spends context on a capture that can't run here, and redirect it to "capture elsewhere, digest here."

The four feedback channels (v1.2 — the Development Observatory)

Every change a session makes propagates through the same four stages, each with its own verbose artifact: what changed (RepoState) → does it build, at what cost (BuildDigest) → does it pass (CIDigest) → how fast is it (PerfDigest). perfdigest digests all four through the same seven tools — a backend is a registry row, not a new API surface. One principle binds the whole matrix: we bind to artifacts, never to live APIs — every reader parses a file on disk whose format is frozen by its ecosystem's backward-compat pressure. (CI artifacts need no dedicated backend at all: a downloaded .ncu-rep from a CI runner is digested by nsight — capture anywhere, digest anywhere, by composition.)

GPU backends share one vocabulary (compute_pct_peak, dram_pct_peak, l2_hit_rate, achieved_occupancy, …); the CPU backend introduces a CPU vocabulary (ipc, cache_miss_rate, llc_miss_rate, branch_mispredict_rate, self_pct, …). The ptxas backend adds the codegen layer: runtime counters say what is slow, registers_per_thread / spill_stores_bytes / static_smem_bytes say why the code became what it is — captured at compile time, so it works in any CI job with no GPU attached.

RepoState is deliberately session-retrospective, not GitHub-shaped: a saved numstat snapshot is the token-cheap answer to "what has this session changed so far", and it doubles as the provenance anchor that pins perf/build/CI reports to a code state. It emits facts (files, line counts, honest None for binary files) — never verdicts like "release ready"; concluding is the agent's job. Future: Go, Java, and other perf-critical runtimes.

Two load-bearing invariants (read before running)

1. Suppress profiler stdout — write to a file. e.g. ncu --set full -o report.ncu-rep ./app. If the profiler prints its summary to stdout, that raw table enters the agent's context before perfdigest runs — defeating the entire purpose.
2. None means "not measured in this export", NEVER zero. A metric the export does not contain is returned as not_available_in_this_export, not a fake 0.0. A genuine 0.0 (e.g. zero branch divergence) is preserved. Silently returning zero = lying to the model = the worst bug this tool can have.

Tools

Tier 1 — digest (any backend, any host):

• summarize_report(report_ref, format, top_n=5) → the N hottest units + core metrics in one call (ranked by duration_us, falling back to self_pct, then file order; reports duration coverage of the returned units)
• list_kernels(report_ref, format) → [{name, index, duration_us, domain}]
• get_metrics(report_ref, format, kernel, metrics=None) → compact digest (metrics=None → the backend's default core set)
• compare_metrics(report_a, report_b, format, kernel, kernel_b=None) → the measure→edit→measure tool: {a, b, delta, delta_pct} per metric with delta = b − a; a metric missing on either side yields an honest not_available_in_this_export delta, never a fake 0.0
• expand(report_ref, format, kernel, section) → raw vendor metrics (the safety valve)

Reports are parsed once per file version (an mtime/size-keyed cache), so repeated digests of the same report cost no re-parse.

Tier 2 — capture advisory (platform-verified):

• platform_capabilities() → machine identity + can_digest (universal) vs can_capture_here (gated)
• suggest_profile_command(backend, target) → the correct, platform-aware invocation, or a refusal that redirects to the tier-1 path

format is mandatory — the agent passes what it produced; a path says where a file is, not what format it is.

Install & connect

uvx perfdigest-mcp                      # run from PyPI (downloadable)
uv tool install "perfdigest-mcp[nvidia]"  # + NVIDIA native binary reader (Linux/Windows)

PyPI/install name is perfdigest-mcp; the command and import package are perfdigest (e.g. uvx perfdigest-mcp, import perfdigest).

Claude Code and OpenAI Codex setup (both stdio MCP): see docs/clients.md.

Status

v1.2.1 — MCP SDK v1 compatibility hotfix

Fresh installs now require mcp[cli]>=1.2,<2, preserving the FastMCP interface used by this release when MCP SDK 2 is available. The seven tool names and parameters and the Python 3.10+ support range are unchanged. This release also packages fixes already merged after v1.2.0 in #32 and #33: torch.compile envelopes carry [dispatch] tags; summaries expose distinct-name coverage and use math.fsum for stable totals; comparisons add domain_a and domain_b while retaining the existing domain alias. Usage guidance explains fusion/group comparisons, Ninja time origins and names omitted by top-N. Validation: 216 tests pass with 16 real-report fixture skips; the existing Windows 3.14t pywin32==312 wheel gap remains the sole allowed CI failure. Clean wheel installs and actual stdio calls to all seven tools were checked on Python 3.10 and 3.14 with MCP 1.30.0, as well as the locked MCP 1.27.2 environment. The declared MCP 1.2 floor is retained: forcing that old SDK with current Pydantic fails on a removed private import, while its compatible Pydantic 2.10.6 and pydantic-settings 2.7.1 combination passes. Normal fresh installs select a recent SDK 1 release.

v1.2.0 — the Development Observatory release (Python 3.10–3.14 incl. free-threaded 3.14t, CI-verified on Linux/macOS/Windows; one known upstream gap: Windows 3.14t cannot install yet because mcp's pywin32 dependency ships no cp314t wheel — that CI cell is a documented allowed-failure until pywin32 catches up): seven new backends extend the digest matrix from profilers to the whole edit→build→verify→measure loop. BuildDigest: clang_time_trace (compiler-phase traces, with [total] aggregate tagging), cmake_profile (configure-step traces — cmake emits bare B/E event pairs, stack-folded honestly), ninja_log (per-edge build timings, v5–v7 logs), cargo_diag (rustc diagnostics — a backend whose durations are honestly absent), criterion (Rust benchmark estimates; first directory-shaped report ref). CIDigest: gha_log (saved gh run view --log files, error/warning annotations, previous-green comparison proven on real two-run fixtures). RepoState: git_numstat (session-retrospective change digest; binary files surface as honest absence, never 0.0). Zero new tools — every backend rides the same seven; 206 tests. Every fixture in the release was produced by the real tool it mimics — several briefs were corrected by what the artifacts actually contained.

v1.1.1 — contract-hardening hotfix (issues #2/#4/#5): registry format collisions are detected on the normalized key (a case-variant format can no longer silently hijack a backend); metrics=[] now means an empty request (only None selects the default core set); exact unit names win over numeric index interpretation, with #3 as the explicit index form; unreadable backends surface a diagnostic in platform_capabilities (can_digest_issues) instead of a silent False; and publishing is gated — a tag only reaches PyPI after the full suite, a clean-venv wheel install, and a tool-surface check pass.

v1.1.0 — the agent-loop release: compare_metrics (before/after deltas), summarize_report (one-call top-N), a parse-once report cache, the ptxas codegen backend (compile-time registers/spills/smem; capture needs nvcc, not a GPU), and the chrome_trace backend (torch/Kineto and any Chrome-trace emitter: framework ops + the CUDA kernels they launched, one report). Also normalizes perf scope-modifier event names (cycles:u on perf_event_paranoid>=2 hosts).

v1.0.0 — multi-backend registry; NVIDIA (native + CSV), AMD HIP, Linux perf (C++/Rust), and Apple Metal adapters; platform capability gating; cross-client config. Validated on the Linux/macOS/Windows CI matrix (pure-Python readers run hardware-free against committed fixtures). Real binary-capture tests are fixture-gated and skip without the device.

Measured behavior

Every number here comes from a real capture on real hardware, published with the workload that produced it in PerfDigest-MCP-Bench (local eval/README.md is the pointer). Ratios are digest payload vs the raw report an agent would otherwise read.

The low ratios are published next to the high ones on purpose. A .ninja_log digests at 0.80× and its summarize_report at 0.49× — a build log is already terse, so there is nothing to compress; what the digest buys there is a uniform contract, not fewer tokens. A measurement tool that only reported its wins would not be one.

Related / similar projects

perfdigest was authored independently; these adjacent MCP servers occupy a nearby space and likely work well for their narrower scope. The differences are the reason perfdigest exists:

• nsys-mcp (NVIDIA Nsight Systems) — profiles binaries and aggregates trace timeline stats. perfdigest targets Nsight Compute per-kernel counters, is read-only (does not run the profiler), and spans multiple vendors.
• pprof-analyzer-mcp / Profiler-MCP — Go (and Go/Python/Java) CPU/memory profiles, often rendering flamegraphs. perfdigest is a numeric digester focused on token/context efficiency rather than visualization.

What is distinct here: the multi-backend matrix (NVIDIA + AMD HIP + CPU perf + Metal under one neutral contract), the token-efficiency thesis with the None≠0.0 honesty rule, and the read/capture split with platform capability gating (digest anywhere, capture only where supported).

License

Apache-2.0 — see LICENSE and NOTICE.