Kubeflow MCP Server

Proposal: KEP-936 · ROADMAP · SECURITY · CONTRIBUTING

Overview

The Kubeflow MCP Server exposes Kubeflow Training operations as Model Context Protocol tools, enabling AI agents (Claude, Cursor, Claude Code, or any custom agents etc.) to plan, submit, monitor, and manage training jobs through natural language — without users needing to learn Kubernetes or the Kubeflow SDK directly.

Benefits

• Agent-Native: Tools auto-discovered via MCP — no manual API wiring
• Guided Workflow: Phase ordering with next-step hints (Plan → Discover → Train → Monitor)
• Preview-Before-Submit: Every mutating operation requires explicit confirmation
• Security-First: Persona gating, namespace enforcement, input validation, bearer/JWT auth
• Multi-Platform: Auto-detects OpenShift, EKS, GKE with platform-specific guidance
• Token-Efficient: Progressive/semantic modes compress 23 tools into 2-3 meta-tools
• Extensible: Plugin architecture for additional Kubeflow clients (TODO: optimizer, hub)

Demo

Get Started

Install from source

git clone https://github.com/kubeflow/mcp-server.git
cd mcp-server
pip install .

Run the server

kubeflow-mcp serve

Once published to PyPI, install with pip install kubeflow-mcp.

Run with Docker

Pre-built multi-arch images are published to GHCR on every release:

docker run --rm -p 8000:8000 \
  -e KUBEFLOW_MCP_AUTH_TOKEN=my-secret-token \
  ghcr.io/kubeflow/mcp-server:latest

The server listens on http://localhost:8000/mcp.

Container and Kubernetes probes are available without MCP authentication:

GET /health  # liveness: the server process is accepting HTTP requests
GET /ready   # readiness: configured clients imported and packaged resources loaded

/ready returns 200 only when both clients_ready and resources_ready are true. It does not contact Kubernetes or other APIs, so it is not a live dependency check. A missing packaged resource Markdown file keeps /ready at 503 even though /health and registered tools remain available; check the server logs and package contents rather than cluster dependencies.

Environment variables

MCP client config (HTTP transport)

{
  "mcpServers": {
    "kubeflow": {
      "url": "http://localhost:8000/mcp",
      "headers": { "Authorization": "Bearer my-secret-token" }
    }
  }
}

For in-cluster deployments, replace localhost:8000 with the Kubernetes Service address and mount KUBEFLOW_MCP_AUTH_TOKEN from a Secret.

Note: DNS rebinding protection allows only loopback Host/Origin headers by default. When exposing the server through a Service or Ingress, set KUBEFLOW_MCP_ALLOWED_HOSTS (e.g. KUBEFLOW_MCP_ALLOWED_HOSTS=kubeflow-mcp.kubeflow.svc:*,mcp.example.com) or requests will be rejected with HTTP 421.

Example: Fine-tune a model via AI agent

Once connected, your AI agent can run a complete training workflow through natural language:

User: "Fine-tune gemma-2b on the alpaca dataset"

Agent calls: check_compatibility()        → ✅ K8s 1.29, Trainer CRD installed
Agent calls: get_cluster_resources()      → 4x A100 GPUs available
Agent calls: estimate_resources("google/gemma-2b") → needs ~16GB GPU, 1x A100
Agent calls: list_runtimes()              → torchtune-llama, torchtune-gemma, ...
Agent calls: fine_tune(                   → preview config (confirmed=False)
    model="hf://google/gemma-2b",
    dataset="hf://tatsu-lab/alpaca",
    runtime="torchtune-gemma-2b"
)
Agent calls: fine_tune(..., confirmed=True) → TrainJob "train-gemma-abc" created
Agent calls: get_training_logs("train-gemma-abc") → training progress...

Every mutating tool requires confirmed=True — agents always preview before submitting.

MCP Client Config

{
  "mcpServers": {
    "kubeflow": {
      "command": "uv",
      "args": ["run", "kubeflow-mcp", "serve"]
    }
  }
}

claude mcp add kubeflow -- kubeflow-mcp serve

Tools

23 tools organized by workflow phase:

Requirements

CLI Reference

kubeflow-mcp serve

kubeflow-mcp serve \
  --clients trainer \             # modules: trainer, optimizer (stub), hub (stub)
  --persona ml-engineer \         # readonly | data-scientist | ml-engineer | platform-admin
  --mode full \                   # full | progressive | semantic
  --instruction-tier full \       # full | compact | minimal
  --transport stdio \             # stdio | http | sse
  --auth-token SECRET \           # bearer token for HTTP auth (dev/staging)
  --otel-endpoint URL \           # OTLP HTTP endpoint (optional tracing)
  --log-level INFO \              # DEBUG | INFO | WARNING | ERROR
  --log-format console \          # console | json (auto-detected if omitted)
  --no-banner                     # suppress startup banner

--mode progressive exposes 3 meta-tools (~85 tokens) for hierarchical discovery. --mode semantic exposes 2 meta-tools (~69 tokens) using embedding search. Both reduce token consumption significantly for agent workflows.

# Simple API key (dev/staging)
kubeflow-mcp serve --transport http --auth-token my-secret-token

# Or via env var
export KUBEFLOW_MCP_AUTH_TOKEN=my-secret-token
kubeflow-mcp serve --transport http

# JWT verification (production)
export KUBEFLOW_MCP_JWKS_URI=https://auth.example.com/.well-known/jwks.json
export KUBEFLOW_MCP_JWT_ISSUER=https://auth.example.com
export KUBEFLOW_MCP_JWT_AUDIENCE=kubeflow-mcp
kubeflow-mcp serve --transport http

kubeflow-mcp agent \
  --backend ollama \              # ollama (default; more backends planned)
  --model qwen3:8b \              # model name for the backend
  --mode full \                   # full | progressive | semantic
  --thinking                      # enable thinking output (supported models)

Observability

OpenTelemetry tracing is optional and can be enabled without changing tool code.

• Install optional dependencies: pip install ".[otel]"
• Enable tracing with CLI flag or env var:

kubeflow-mcp serve --otel-endpoint http://localhost:4318
# or
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
kubeflow-mcp serve

Each tool invocation emits a span with attributes: tool.name, tool.args_preview, tool.success, tool.duration_ms, kubeflow.persona, and correlation_id.

Note: kubeflow-mcp agent --otel-endpoint ... emits spans under a separate kubeflow-mcp-agent service in Jaeger, distinct from the kubeflow-mcp server spans.

Development

make install-dev                  # setup environment
make verify                       # lint + format check
make test-python                  # run tests
make inspector                    # launch MCP Inspector (stdio)
make inspector TRANSPORT=http     # Inspector + Streamable HTTP (start server separately)
make inspector TRANSPORT=sse      # Inspector + SSE (start server separately)

Community

• Slack: Join #kubeflow-ml-experience on CNCF Slack
• Meetings: Attend the Kubeflow SDK and ML Experience bi-weekly call
• GitHub: Issues and contributions at kubeflow/mcp-server

Documentation

• CONTRIBUTING: Development workflow and PR guidelines
• ROADMAP: Project roadmap
• SECURITY: Vulnerability reporting; see ARCHITECTURE.md#security-model for threat model, RBAC, and hardening
• KEP-936: Design proposal

License

Apache License 2.0 — see LICENSE.