agent-capmap

Find out what your AI agents can actually do, and which combinations of their tools an attacker could abuse.

AI assistants such as Claude Desktop, Cursor, VS Code and Claude Code get their powers from MCP servers: read files, browse the web, send email, run shell commands. Each tool can look harmless on its own. The danger is usually the combination: an agent that can read private data, read attacker-written content (a web page, an issue, an email), and send data out can be talked into leaking that data. That is the "lethal trifecta".

agent-capmap inventories every MCP server and tool reachable by each agent, classifies what each tool can do, and applies policy-as-code rules to the combination of tools per agent. Findings come with evidence and can gate CI through SARIF and exit codes.

Status: pre-release (0.1.0). All features below work. The classifier's accuracy has not been benchmarked yet; see Accuracy.

Install

Requires Python 3.11+.

pipx install agent-capmap        # or: uv tool install agent-capmap

Releases are built and published by GitHub Actions with PyPI trusted publishing, so each file on PyPI carries an attestation linking it to the workflow run that built it.

For a fully hash-pinned install, with the exact dependency versions CI tests, install from a clone instead:

git clone https://github.com/NKwolv/agent-capmap
cd agent-capmap
python -m venv .venv
. .venv/bin/activate            # Windows PowerShell: .venv\Scripts\Activate.ps1
pip install --require-hashes -r requirements/runtime.lock
pip install --no-deps .

Locked-down Windows machines: if your organization's policy (Device Guard, AppLocker, Smart App Control) blocks agentcapmap.exe or pipx.exe, run the tool through Python instead, which is the same program. If you installed with pip install agent-capmap, run python -m agentcapmap scan --user. If you installed with pipx, the package lives in pipx's own environment, so plain python will not find it: use that environment's interpreter (python -m pipx environment --value PIPX_LOCAL_VENVS shows the folder): <venvs>\agent-capmap\Scripts\python.exe -m agentcapmap scan --user. Do not try to bypass the policy; ask your administrator to allow the launcher if you want it.

Windows: if pip install fails with "No such file or directory" deep inside site-packages, the path is too long for Windows' 260-character limit. Clone to a short path (for example C:\src\agent-capmap) or enable long path support.

Quickstart (5 minutes, no API key)

# 1. See findings on a ready-made inventory of five example agents
#    (from a clone, use examples/demo/inventory.json directly)
curl -fsSLO https://raw.githubusercontent.com/NKwolv/agent-capmap/main/examples/demo/inventory.json
agentcapmap evaluate inventory.json --format markdown

# 2. List the MCP servers configured on your own machine (reads files only)
agentcapmap scan --user

On Windows PowerShell, use curl.exe instead of curl.

Step 1 should report 2 critical, 2 high and 1 medium finding. Step 2 lists your servers with every secret redacted, without starting anything.

Example output

### 1. `CAP-001` | Critical

`Lethal trifecta: the agent can read private data, ingest attacker-controlled content,
and send data externally, so injected instructions can exfiltrate data.`

- **Agent:** `generic:fixtures/trifecta/mcp.json` at `fixtures/trifecta/mcp.json:3`
- **Tools:** `github/get_file_contents`, `github/list_issues`, `mailer/send_email`

| Server | Tool | Matched | Evidence |
|---|---|---|---|
| `github` | `get_file_contents` | `data_access=private_read` | `contents of a file from a private or public repository` |
| `github` | `list_issues` | `untrusted_input=true` | `including issues opened by anyone` |
| `mailer` | `send_email` | `egress=external` | `Send an email to any recipient address` |

(Abridged from the real output of step 1.)

How it works

config files ──► discover ──► enumerate ──► classify ──► policy ──► report
                 (read only)  (initialize +  (LLM tags    (YAML      (JSON, Markdown,
                              tools/list)    6 axes)      rules)     SARIF, exit code)

1. Discover MCP configs: Claude Desktop, Cursor, VS Code (.vscode/mcp.json, settings.json), Claude Code (.mcp.json, and with --user your ~/.claude.json user and per-project servers, including permission rules) and generic mcp.json. Secrets are redacted everywhere.
2. Enumerate (--enumerate): start each server and ask only for its tool list. Tools are never called.
3. Classify (--classify): an LLM tags each tool on six axes and flags tool descriptions that try to manipulate the model. Results are cached and validated against a JSON schema.
4. Evaluate (--evaluate): deterministic rules decide what is dangerous. The LLM never makes risk decisions.
5. Report: JSON, Markdown, or SARIF for GitHub code scanning; --fail-on sets the exit code.

Full scan of your own setup (needs an API key)

export ANTHROPIC_API_KEY=<YOUR_API_KEY>          # PowerShell: $env:ANTHROPIC_API_KEY = "..."
agentcapmap scan --user --enumerate --classify --evaluate --format markdown -o report.md

--enumerate runs the server commands in your configs: only use it on configs you trust. To try the whole pipeline on two known servers first, see examples/live-trifecta.

Capability taxonomy

Definitions and edge-case rulings: docs/taxonomy.md.

Policy

The built-in policies/default.yaml has nine rules:

Write your own rules in the same format and pass --policy my-policy.yaml:

- id: ACME-001
  severity: high
  when_server: {remote: true}
  when_agent_has_any:
    - privilege: identity_admin
  message: Remote servers must not manage identities.

Conditions: when_agent_has_all, when_agent_has_any, when_server, when_hash_changed, when_tool_name_shadowed. The format is defined by schemas/policy.schema.json. Rug-pull detection:

agentcapmap baseline create inventory.json -o baseline.json   # once, after review
agentcapmap evaluate new-inventory.json --baseline baseline.json

To see what changed between two runs (for example main versus a pull request), save each evaluate output and compare them:

agentcapmap diff before.json after.json --format markdown
agentcapmap diff before.json after.json --fail-on high --fail-on-new-capability

diff lists new and resolved findings and capabilities an agent gained or lost. --fail-on looks at new findings only, so existing, already-accepted findings do not fail the build; --fail-on-new-capability fails when an agent gains any capability.

CI and GitHub code scanning

agentcapmap evaluate inventory.json --format sarif -o agentcapmap.sarif --fail-on high

Exit codes: 0 passed, 1 findings at or above --fail-on, 2 usage or input error. The repository is also a GitHub Action: see docs/github-action.md and the example workflow. By default the Action reads configs only, which is safe for pull requests from forks.

Accuracy

The classifier is evaluated against hand-labeled tools from real MCP servers (see eval/). No accuracy numbers are published yet. The 381 labels (28 servers) were drafted by one AI model and then reviewed by another AI model at the maintainer's request; no person has independently reviewed them yet. A model grading a model can overstate accuracy, so numbers will be quoted here only after a person has reviewed the labels and the run is frozen (python eval/run_eval.py). The same labels also power classification without an API key for these known tools; see Limitations.

Limitations

• Descriptions, not behavior. Tools are classified by what their definitions say. A tool that lies about itself will be mis-tagged; agent-capmap never calls tools.
• Classifier errors. Tags can be wrong, especially for vague descriptions. Every tag carries a confidence and an evidence phrase; phrases not found in the definition are marked unverified.
• Approval gates are mostly invisible. Claude Code's permission settings are read (so its default "ask before each MCP call" counts as a gate), but most other clients do not record this, and CAP-003 treats "unknown" as "no gate" by default.
• Enumeration runs code. Starting servers executes the configured commands. A started server can also read its parent's environment on Linux.
• Coverage. Config formats evolve; Windsurf is not read yet. MCP servers that do not live in config files (Claude Code plugins, claude.ai connectors, IDE built-in servers, managed settings) are not seen. WebSocket servers are listed but not enumerated. Token scopes are not visible.
• Reference labels are AI-reviewed. For 381 tools from 28 well-known servers, the classifier answers from bundled labels (matched by exact definition hash) instead of calling a model, so no API key is needed. Those labels were reviewed by an AI model, not yet by a person, and a wrong label is returned with full confidence; any change to a tool's definition falls back to the cache or the LLM. Treat them as a convenience.
• Not assessed is not safe. Servers that could not be enumerated and tools that could not be classified are listed under not_assessed, never assumed safe.

Safety

Secrets from configs are redacted in all output and logs. Enumeration sends only initialize and tools/list, with a minimal environment for launched servers, no credential-forwarding redirects, and size limits on everything a server returns. Reports render untrusted text inertly. See SECURITY.md and the threat model.

Documentation

• Capability taxonomy
• Threat model: attacks covered, OWASP / MITRE ATLAS mapping
• GitHub Action
• Comparison with Snyk Agent Scan (mcp-scan)
• Examples · Evaluation · POC checklist
• Contributing · Security policy · Changelog

License

MIT. See LICENSE.