ptm-mcp 0.4.0

MCP stdio server for the Prompt Test Manager API

ptm-mcp

MCP (Model Context Protocol) stdio server for the Prompt Test Manager API.

Lets agents built on top of MCP-capable clients (Claude Desktop, Claude Code, Codex, etc.) call PTM as first-class tools: list prompts, run evaluations, update prompt content, submit optimizations. All traffic is tagged with X-PTM-Client: ptm-mcp/<version> + a per-process X-PTM-MCP-Session UUID so the PTM backend can rate-limit, budget, and audit agent traffic separately from humans and service accounts.

Prereqs

• Python >= 3.12.
• A reachable PTM backend (>= 1.9.0) and a personal access token or service-account token with the scopes your flow needs.

Install

pip install ptm-mcp

Or zero-install via uvx:

uvx ptm-mcp

ptm-mcp pulls in ptm-client and the mcp SDK automatically.

Configure your MCP client

Claude Desktop

Config file:

OS	Path
macOS	~/Library/Application Support/Claude/claude_desktop_config.json
Windows	%APPDATA%\Claude\claude_desktop_config.json

Merge into mcpServers (create the key if it doesn't exist):

{
  "mcpServers": {
    "ptm": {
      "command": "uvx",
      "args": ["ptm-mcp"],
      "env": {
        "PTM_API_BASE_URL": "https://ptm.example.com",
        "PTM_API_TOKEN": "ptm_u_PASTE_HERE",
        "PTM_MCP_READ_ONLY": "true"
      }
    }
  }
}

Fully quit + reopen Claude Desktop after editing. stderr lands in ~/Library/Logs/Claude/mcp-server-ptm.log (macOS) or %APPDATA%\Claude\logs\mcp-server-ptm.log (Windows).

Claude Code

# macOS / Linux
claude mcp add --transport stdio --scope user ptm \
  --env PTM_API_BASE_URL=https://ptm.example.com \
  --env PTM_API_TOKEN=ptm_u_PASTE_HERE \
  --env PTM_MCP_READ_ONLY=true \
  -- uvx ptm-mcp

# Windows (PowerShell / cmd) - needs cmd /c wrapper
claude mcp add --transport stdio --scope user ptm `
  --env PTM_API_BASE_URL=https://ptm.example.com `
  --env PTM_API_TOKEN=ptm_u_PASTE_HERE `
  --env PTM_MCP_READ_ONLY=true `
  -- cmd /c uvx ptm-mcp

Codex

~/.codex/config.toml (macOS / Linux) or %USERPROFILE%\.codex\config.toml (Windows):

[mcp_servers.ptm]
command = "uvx"
args = ["ptm-mcp"]
env = { PTM_API_BASE_URL = "https://ptm.example.com", PTM_API_TOKEN = "ptm_u_PASTE_HERE", PTM_MCP_READ_ONLY = "true" }
startup_timeout_sec = 10
tool_timeout_sec = 60

Or CLI-first: codex mcp add ptm -- uvx ptm-mcp (with --env KEY=VAL per var).

Environment variables

Consumed at startup. Missing required values fail fast with a descriptive error.

Variable	Required	Default	Notes
PTM_API_BASE_URL	yes	-	e.g. https://ptm.example.com
PTM_API_TOKEN	yes	-	PTM bearer. Service-account tokens preferred for long-running agent sessions.
PTM_MCP_READ_ONLY	no	true	Flip to false to unlock write tools.
PTM_MCP_TIMEOUT_SECONDS	no	30	Per-request timeout (1..600).
PTM_MCP_LOG_LEVEL	no	INFO	DEBUG / INFO / WARNING / ERROR / CRITICAL.
CF_ACCESS_CLIENT_ID	no	-	Cloudflare Access service-token Client ID. Paired with CF_ACCESS_CLIENT_SECRET.
CF_ACCESS_CLIENT_SECRET	no	-	Cloudflare Access service-token Client Secret. Paired with CF_ACCESS_CLIENT_ID.
CF_ACCESS_JWT	no	-	Cloudflare Access user JWT (alternative to the service-token pair).
PTM_CF_AUTO_DISCOVER	no	true	Falsy value opts out of auto-discovery via cloudflared.

Startup scrubs every env var outside a narrow allow-list (cloud creds, GitHub tokens, etc. get dropped).

Cloudflare Access

If your PTM deployment sits behind Cloudflare Access:

• Default path (recommended): install cloudflared (brew install cloudflared or equivalent) and run cloudflared access login https://your-ptm-host once. ptm-mcp auto-detects CF challenges and injects the cached JWT on request.
• Service token (CI / headless): ask an admin to mint a service token for the PTM app in Cloudflare Zero Trust -> Access -> Service Auth. Set CF_ACCESS_CLIENT_ID + CF_ACCESS_CLIENT_SECRET in the MCP env block. Explicit config disables auto-discovery.
• Direct access (no CF Access): skip this section; no CF env vars needed.

On a Cloudflare Access block, ptm-mcp surfaces a CloudflareAccessError with the exact next step rather than a raw JSON decode crash.

Tool inventory

24 tools total (1 canary + 12 read + 11 write) + 5 resource URI patterns. Write tools are gated by PTM_MCP_READ_ONLY=false; the backend enforces per-prompt ownership, group membership, and admin role on top of the PAT's scopes. ptm-mcp relays backend errors verbatim with actionable hints; it does not re-implement permission checks.

Read (13)

list_providers, list_prompts, get_prompt, get_prompt_tests, list_prompt_versions, get_prompt_version, compare_prompt_versions, list_runs, get_run, get_run_report, get_optimization_status, get_optimization_history, get_optimization_detail.

Write (11, gated by PTM_MCP_READ_ONLY)

Eval / optimization (4): run_manual_eval, run_prompt_eval, submit_optimization, cancel_optimization.

submit_optimization accepts seven optional variance-aware fields as of 0.4.0 / PTM 1.10.0: stability_samples (1-10), validation_samples (1-10), flakiness_threshold (0.0-1.0), min_consistent_improvement (0.0-100.0), variance_aware_mutator (bool), variance_signal (levenshtein / embedding / both), enforce_target_score (bool, default true). Omit any field to inherit the server's admin-configured default.

Library mutation (7):

• update_prompt - unified atomic update. Partial-update semantics: null/omitted = leave unchanged, [] = clear a list, populated list = replace. Content fields (prompt_text, tests, deepeval_metrics, kpis, provider_profiles, judge_profile) auto-create a new version. Required change_summary flows into the audit log.
• activate_prompt_version - PUT /versions/{n}/activate. Rollback / roll-forward path with no content change.
• share_prompt, unshare_prompt, add_prompt_to_group, remove_prompt_from_group - group-scoped visibility. Admin or group-manager roles only.
• transfer_prompt_ownership - admin only. Current owner cannot self-transfer.

Permissions model

Action	Who can do it
Read tools	Anyone with a valid PAT (results filtered by visibility scope)
update_prompt, activate_prompt_version	Prompt owner OR a role with prompt-overwrite permission
share_prompt, unshare_prompt, add_prompt_to_group, remove_prompt_from_group	Admin OR group manager
transfer_prompt_ownership	Admin ONLY
Eval tools	Anyone who can read the prompt

Backend error codes surface as actionable tool errors:

• 403 -> "Permission denied. This operation requires prompt ownership or admin / group-manager role."
• 404 -> "Prompt or resource not found. Verify prompt_id and PAT visibility."
• 409 -> "Conflict. Repository-backed prompt (edit the files at source) OR concurrent writer. Re-fetch and retry."
• 422 -> "Validation failed. Check prompt_text length (max 500k), test / metric shapes, and field types."

Resources (5 URI patterns)

• ptm://prompts/{prompt_id} - active version's prompt_text (text/plain)
• ptm://prompts/{prompt_id}/v{N} - that version's prompt_text (text/plain)
• ptm://runs/{run_key}/report.md - markdown report (text/markdown)
• ptm://runs/{run_key}/report.html - HTML report (text/html)
• ptm://optimizations/{optimization_id}/report.md - markdown summary (text/markdown)

Dynamic segments are allow-list validated (^[a-zA-Z0-9_.-]+$ plus explicit ./.. rejection).

Security defaults

• PTM_MCP_READ_ONLY=true blocks every write tool at call time.
• X-PTM-Client + X-PTM-MCP-Session on every outbound request so the backend can classify and audit agent traffic.
• Env scrub at startup drops anything outside the allow-list.
• Startup preflight (/healthz + /auth/me + /meta) with exponential backoff on transient failures and dedicated exit codes per failed layer.

Exit codes

Code	Meaning
0	clean shutdown
1	unhandled exception
2	/healthz unreachable after 31s of backoff
3	/auth/me rejected the token
4	backend version < 1.9.0 or unparseable
130	interrupted (SIGINT)

Status

0.2.2. Stable tool surface: 24 tools + 5 resource URI patterns, read-only gate, Cloudflare Access auto-discovery, unified update_prompt. See CHANGELOG.md for release notes.