AgentKit — A Governed MCP Tool Server

An MCP server where tools are declarative, effects are typed, and every action is policy-gated and audited — usable by any MCP client (Claude Desktop, Cursor, LangGraph, Claude Agent SDK, CrewAI).

Live Application: agentkit.ysiddo-ai-projects.app.

• Research Background: RESEARCH.md / Online Research Documentation
• Empirical Benchmarks: BENCHMARK.md / Online Benchmark Dashboard
• User Guide: Online User Guide
• Self-Hosting Guide: SELF_HOSTING.md

Three things distinguish it from a typical MCP server:

• Declarative tools — define tools in YAML over your own Postgres or HTTP API. No Python, no fork. (docs/REUSE.md)
• Real actions, not just reads — tools declare an effect (read / write / destructive) and mutating tools genuinely mutate.
• Guardrails that hold regardless of the prompt — writes are off by default, destructive actions need a human-held approval token the model never sees, everything supports dry-run, and every call (allowed and denied) is audited. (SECURITY.md)

The bundled business-intelligence tools below are the reference pack that demonstrates all of this — not the limit of what the server does.

What It Does

Reference BI pack (built in):

• 6 core MCP tools: query_kpis, get_company_health, detect_kpi_anomalies, forecast_metric, list_available_metrics, get_executive_summary — plus any tools loaded from declarative YAML packs (packs/), so the real tool count at runtime is 6 or more, not a fixed number. The bundled annotations pack adds 3 more (list_annotations, annotate_metric, retract_annotation) demonstrating a guarded write/destructive path — see below.
• 10 MCP resources: kpi://{domain}/latest for each of this project's real seeded domains — Finance, People, Operations, Customer, Engineering, Growth, Logistics, ESG, IT, Security (src/agentkit_mcp/data/seed.py).
• 1 reusable prompt: monthly_executive_briefing

The domain list, tool-pack set, and resource/prompt registration are all derived from the same data this server serves (src/agentkit_mcp/data/seed.py, packs/*.yaml), not hardcoded, so adding a domain or tool pack extends this list automatically. You can also add your own @mcp.resource / @mcp.prompt decorators. See docs/REUSE.md.

Write and destructive actions are real, not aspirational. The annotations pack lets an agent record a durable note on a metric (annotate_metric, effect write) and soft-delete one it got wrong (retract_annotation, effect destructive — requires a human-held AGENTKIT_APPROVAL_TOKEN the model never sees). Both are off by default (AGENTKIT_ALLOW_WRITES=false) — see SECURITY.md and RESEARCH.md §2 for the full policy-engine model.

Platform capabilities:

• Declarative tool packs — add tools over your own Postgres/HTTP in YAML (packs/)
• Typed effects + policy engine — GET /api/policy publishes the capability envelope
• Audit trail — GET /api/audit, allowed and denied, with deny reasons
• Multi-provider LLM routing incl. self-hosted — GET /api/llm-routing
• LangGraph 3-agent workflow in workflow.py (Planner → Analyst → Reporter)
• Claude Agent SDK demo in demos/claude_agent_sdk_demo.py
• CrewAI demo in demos/crewai_demo.py
• DSPy research scaffold in research/dspy_experiment.py
• 67 tests across smoke, API, integration, MCP protocol, policy guardrails, and LangGraph workflow

PyPI Package

pip install agentkit-mcp   # v0.1.15
agentkit-mcp               # CLI entrypoint

Quick Start

python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env  # fill in keys + POSTGRES_URL
python mcp_server.py

Claude Desktop Setup

Add to ~/.config/Claude/claude_desktop_config.json:

{
  "mcpServers": {
    "agentkit": {
      "command": "python",
      "args": ["/abs/path/to/agentkit/mcp_server.py"],
      "env": {
        "MCP_TRANSPORT": "stdio",
        "POSTGRES_URL": "postgresql://...",
        "LOG_LEVEL": "DEBUG",
        "TELEMETRY_OPT_OUT": "true"
      }
    }
  }
}

MCP_TRANSPORT=stdio is required here — without it mcp_server.py defaults to serving over SSE (a network port) instead of talking JSON-RPC over the pipes Claude Desktop spawns it with, and no tools will appear. Local stdio mode doesn't need MCP_AUTH_TOKEN (the OS process boundary is the auth boundary); that variable only matters for the SSE/network path.

Multi-Provider LLM Routing

The 3-agent LangGraph workflow (workflow.py) and the demos/research scripts route each role to its own model via LiteLLM, configured with plain provider/model strings — no code changes to switch providers:

• LLM_REASONING — planner + reporter agents (defaults to anthropic/claude-sonnet-4-6)
• LLM_DEFAULT — the tool-calling analyst agent (defaults to groq/openai/gpt-oss-120b)
• LLM_JUDGE — used by the eval suite (defaults to anthropic/claude-haiku-4-5)
• LLM_LOCAL + INFERENCE_MODE=local — route to a local/self-hosted model (e.g. Ollama) instead of a hosted provider

Set the matching provider API key(s) (GROQ_API_KEY, ANTHROPIC_API_KEY, OPENAI_API_KEY) for whichever models you reference above. See .env.example.

• Diagnostics: adjust LOG_LEVEL to DEBUG for verbose logs.
• Telemetry: off by default (TELEMETRY_URL is blank out of the box). Set TELEMETRY_URL to opt in to a single anonymous startup ping (at most once per ~6 hours per instance — a timestamp plus a randomly generated, non-hardware-derived install ID; no prompts, tool calls, or application data), or set TELEMETRY_OPT_OUT=true to disable it outright regardless of TELEMETRY_URL.

Restart Claude Desktop, then ask:

• "What's our company health right now?"
• "Forecast revenue for the next 6 months."
• "Are there anomalies in the Finance KPIs?"

LangGraph Workflow

from agentkit_mcp.workflow import analyze
result = analyze("What drove gross margin in Q1?")
print(result["report"])

Architecture

        Claude Desktop / Cursor / LangGraph
                      │
                      ▼ MCP
              ┌────────────────────────┐
              │     mcp_server.py      │
              │  6 core tools (read)   │
              │  + N pack tools        │
              │    (read/write/        │
              │     destructive)       │
              │ 10 resources, 1 prompt │
              └────────────┬───────────┘
                            │
        ┌───────────────────┼──────────────────┐
        ▼                   ▼                  ▼
   pg_store            insights          forecasting
   (KPIs, real         (health,          (scikit-learn
   domains + write-    anomalies)        LinearRegression
   back annotations)                     + Monte Carlo CI)

Research Contribution

• Standardized MCP middleware — unified stdio and SSE transport for hot-swappable agent tools.
• Capability policy engine — formal effect separation (read/write/destructive) and prompt-independent guardrails.
• Multi-agent interoperability — verified across Claude Desktop, Cursor IDE, and Devin AI.

For the full formalization, literature context, and citation details, see RESEARCH.md.

Benchmark Replication Suite

Run the reproducible benchmark evaluation suites:

# Test MCP framework overhead
python3 eval/run_benchmarks.py --seed 42

# Full 43-scenario LangGraph suite across all 10 KPI domains
python3 eval/run_dspy_eval.py

# Comprehensive MCP tool execution metrics
python3 eval/run_mcp_tools_benchmark.py

Integration Guides (Claude Desktop, Cursor, Devin)

• Claude Desktop: See claude_desktop_config.example.json and docs/INTEGRATION_GUIDE.md
• Cursor IDE: See cursor_mcp.example.json
• Devin AI Agent: See devin_mcp.example.json

Automated client verification:

python3 tests/test_mcp_client.py

License & Enterprise Use (Dual-License)

This project is open-source under the AGPL-3.0 License. It is completely free for researchers, students, and open-source hobbyists. Commercial license: see COMMERCIAL.md.