ptm-mcp 0.1.0

MCP stdio server for the Prompt Test Manager API

ptm-mcp

MCP (Model Context Protocol) stdio server for the Prompt Test Manager API.

Lets agents built on top of MCP-capable clients (Claude Desktop, Claude Code, Codex, Goose) call PTM as first-class tools: list prompts, run evaluations, submit optimizations. All traffic is tagged with X-PTM-Client: ptm-mcp/<version> + a per-process X-PTM-MCP-Session UUID so the PTM backend can rate-limit, budget, and audit agent traffic separately from humans and service accounts.

Quickstart

# 1. Mint a PTM token (prompts for your PTM URL + email + password)
bash scripts/mint-ptm-mcp-token.sh     # macOS / Linux
pwsh scripts/mint-ptm-mcp-token.ps1    # Windows

# 2. Wire into your MCP client of choice
bash scripts/install-claude-desktop.sh # Claude Desktop (macOS / Linux)
pwsh scripts/install-claude-desktop.ps1 # Claude Desktop (Windows)

# Claude Code - macOS / Linux
claude mcp add --transport stdio --scope user ptm -- uvx ptm-mcp

# Claude Code - Windows
claude mcp add --transport stdio --scope user ptm -- cmd /c uvx ptm-mcp

# Codex (any OS)
codex mcp add ptm -- uvx ptm-mcp

# Env vars can be set via `--env KEY=VAL` on either command.
# Full setup + token flow: docs/mcp-integration.md

# Uninstall Claude Desktop entry (creates a timestamped backup):
bash scripts/uninstall-claude-desktop.sh       # macOS / Linux
pwsh scripts/uninstall-claude-desktop.ps1      # Windows

Full setup + troubleshooting: docs/mcp-integration.md.

Status

Phase 2 complete: 17 tools (canary + 12 read + 4 write), 5 resource URI patterns, read-only gate, live end-to-end integration. Ships at 0.1.0.

Prereqs

• Python >= 3.12 in a venv you control.
• PTM backend >= 1.9.0 (MCP middleware chokepoint landed in 1.9.0; older backends cannot enforce agent-scoped limits).
• For the stdio smoke test: Node >= 18 (for npx @modelcontextprotocol/inspector) or a global install of the MCP Inspector CLI.

Install

pip install ptm-mcp

Requires Python >= 3.12. ptm-mcp pulls in ptm-client and the mcp SDK automatically. For pinned, runtime-tested versions see pyproject.toml in the release tag.

From source (dev mode)

git clone [e-mail]:15five/prompt-test-manager
cd prompt-test-manager
python3.12 -m venv .venv && source .venv/bin/activate
pip install -e packages/ptm-client
pip install -e "packages/ptm-mcp[dev]"

Dev mode is what CI exercises. When developing locally, wire your MCP client config at PYTHONPATH=packages/ptm-mcp/src:packages/ptm-client/src so edits in src/ are picked up without reinstalling.

Environment variables

Consumed at startup. Missing required values fail fast with a descriptive error.

Variable	Required	Default	Notes
PTM_API_BASE_URL	yes	-	e.g. https://ptm.example.com
PTM_API_TOKEN	yes	-	PTM bearer. Service-account tokens preferred.
PTM_MCP_READ_ONLY	no	true	Flip to false to unlock the 4 write tools.
PTM_MCP_TIMEOUT_SECONDS	no	30	Per-request timeout (1..600).
PTM_MCP_LOG_LEVEL	no	INFO	DEBUG / INFO / WARNING / ERROR / CRITICAL.

Startup scrubs every env var outside a narrow allow-list (cloud creds, GitHub tokens, PATH - all get dropped). See src/ptm_mcp/env.py.

Claude Desktop config snippet

~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%\Claude\claude_desktop_config.json (Windows):

{
  "mcpServers": {
    "ptm": {
      "command": "uvx",
      "args": ["ptm-mcp"],
      "env": {
        "PTM_API_BASE_URL": "http://localhost:8010",
        "PTM_API_TOKEN": "ptm_u_PASTE_HERE",
        "PTM_MCP_READ_ONLY": "true"
      }
    }
  }
}

uvx handles Python provisioning + caching automatically; no PYTHONPATH needed when installed from PyPI. Other clients (Claude Code, Codex): see docs/mcp-integration.md for client-specific wire-up.

Tool inventory

Read (13)

list_providers, list_prompts, get_prompt, get_prompt_tests, list_prompt_versions, get_prompt_version, compare_prompt_versions, list_runs, get_run, get_run_report, get_optimization_status, get_optimization_history, get_optimization_detail.

Write (4, gated by PTM_MCP_READ_ONLY)

run_manual_eval, run_prompt_eval, submit_optimization, cancel_optimization.

Resources (5 URI patterns)

• ptm://prompts/{prompt_id} - active version's prompt_text (text/plain)
• ptm://prompts/{prompt_id}/v{N} - that version's prompt_text (text/plain)
• ptm://runs/{run_key}/report.md - markdown report (text/markdown)
• ptm://runs/{run_key}/report.html - HTML report (text/html)
• ptm://optimizations/{optimization_id}/report.md - markdown summary (text/markdown)

Dynamic segments are allow-list validated (^[a-zA-Z0-9_.-]+$ plus explicit ./.. rejection). See docs/mcp-security.md.

Security defaults

• PTM_MCP_READ_ONLY=true blocks every write tool at call time.
• X-PTM-Client + X-PTM-MCP-Session on every outbound request.
• Env scrub at startup.
• Startup preflight (/healthz + /auth/me + /meta) with exponential backoff on transient failures and dedicated exit codes per failed layer.

Full details: docs/mcp-security.md.

Development

pip install -e packages/ptm-client
pip install -e 'packages/ptm-mcp[dev]'
PYTHONPATH=packages/ptm-mcp/src:packages/ptm-client/src \
  pytest packages/ptm-mcp/tests -q
ruff check packages/ptm-mcp/src packages/ptm-mcp/tests
ruff format --check packages/ptm-mcp/src packages/ptm-mcp/tests

End-to-end smoke (requires a running backend and Node for npx):

PTM_API_BASE_URL=http://localhost:8010 \
PTM_API_TOKEN="ptm_u_..." \
bash scripts/smoke_mcp_inspector.sh

Exit codes

Code	Meaning
0	clean shutdown
1	unhandled exception
2	/healthz unreachable after 31s of backoff
3	/auth/me rejected the token
4	backend version < 1.9.0 or unparseable
130	interrupted (SIGINT)

See also

• docs/mcp-integration.md - end-user setup
• docs/mcp-developer.md - adding a tool
• docs/mcp-security.md - tokens, env, headers, read-only, kill switch
• docs/mcp-admin.md - ops runbook + alert response