Kafka Mcp

API | MCP Server | A2A Agent

Apache Kafka API + MCP Server + A2A Agent for the Arbora ecosystem.

Version: 0.5.0

Documentation — Installation, deployment, usage across the API, CLI, and MCP interfaces, and guidance for provisioning the Apache Kafka platform are maintained in the official documentation.

kafka-mcp wraps the Apache Kafka administration and data-plane surface with typed, deterministic tools an agent or policy router calls. It speaks to the Confluent REST Proxy by default and ships an optional native (direct-to-broker) client and a Pydantic-AI agent server.

What it provides

• KafkaApi (kafka_mcp.api.api_client_kafka) — a requests-based REST facade over the Confluent REST Proxy v3 (with v2 consumer helpers), organized by Kafka resource: clusters, topics, partitions, records, consumer groups, brokers, and ACLs. The active cluster id resolves lazily unless KAFKA_CLUSTER_ID pins it.
• Six MCP tools (kafka-mcp console script): kafka_topics, kafka_partitions, kafka_records, kafka_groups, kafka_cluster, and the optional native kafka_native. See docs/overview.md for the full action surface.
• An A2A agent server (kafka-agent console script) — a Pydantic-AI graph agent wired to the MCP server via MCP_URL.

Environment Variables

Package environment variables

Inherited agent-connector-sdk variables (apply to every connector)

17 package + 23 inherited variable(s). Auto-generated from .env.example + the shared agent-connector-sdk set — do not edit.

Connection & credentials (Confluent REST Proxy)

Native client (optional kafka-mcp[native] extra)

MCP server / transport

Tool toggles

Copy .env.example to .env and populate only what the operator use; blank connector credentials leave the corresponding surface inactive.

Installation

Pick the extra that matches what the operator want to run:

# Connector-focused MCP server (includes the shared graph engine)
uv pip install "kafka-mcp[mcp]"

# Agent runtime (adds model orchestration to the shared graph engine)
uv pip install "kafka-mcp[agent]"

# Everything (development)
uv pip install "kafka-mcp[all]"      # or: python -m pip install "kafka-mcp[all]"

kafka-mcp                        # stdio MCP server (default transport)
kafka-mcp --transport streamable-http --host 0.0.0.0 --port 8000

Run the agent server against a live MCP server:

kafka-agent --mcp-url http://localhost:8000/mcp --host 0.0.0.0 --port 8080

Container images (:mcp vs :agent)

One multi-stage docker/Dockerfile builds two right-sized images, selected by --target:

docker build --target mcp   -t example/kafka-mcp:mcp    docker/   # connector-focused MCP server
docker build --target agent -t example/kafka-mcp:agent-local docker/   # agent runtime

Knowledge-graph database (epistemic-graph)

Both [mcp] and [agent] carry the epistemic-graph engine through the required Agent Utilities core dependency (epistemic-graph[full]). The [mcp] extra keeps the server connector-focused; [agent] also enables model orchestration. Local deployments can use the bundled engine. For production or shared state, run epistemic-graph as a dedicated database service and configure the runtime to use it. Deployment recipes (single-node + Raft HA), connection configuration, and architecture diagrams are documented in the epistemic-graph deployment guide.

MCP Configuration Examples

Install the connector-focused [mcp] extra. Examples use kafka-mcp[mcp] to add FastMCP / FastAPI through agent-connector-sdk[mcp]; the required Agent Utilities core still carries epistemic-graph[full]. The [agent-runtime] extra additionally enables model orchestration.

stdio Transport (local IDEs — Cursor, Claude Desktop, VS Code)

{
  "mcpServers": {
    "kafka-mcp": {
      "command": "uvx",
      "args": [
        "--from",
        "kafka-mcp[mcp]",
        "kafka-mcp"
      ],
      "env": {
        "KAFKATOOL": "True",
        "KAFKA_BOOTSTRAP_SERVERS": "localhost:9092",
        "KAFKA_CDCTOOL": "True",
        "KAFKA_CONNECTTOOL": "True",
        "KAFKA_CONNECT_URL": "http://localhost:8083",
        "KAFKA_REST_URL": "http://localhost:8082"
      }
    }
  }
}

Runtime references require an alias-aware launcher such as Arbora. Other launchers must omit those entries and inject the resolved values through their own runtime secret boundary.

Streamable-HTTP Transport (networked / production)

{
  "mcpServers": {
    "kafka-mcp": {
      "command": "uvx",
      "args": [
        "--from",
        "kafka-mcp[mcp]",
        "kafka-mcp",
        "--transport",
        "streamable-http",
        "--port",
        "8000"
      ],
      "env": {
        "TRANSPORT": "streamable-http",
        "HOST": "127.0.0.1",
        "PORT": "8000",
        "KAFKATOOL": "True",
        "KAFKA_BOOTSTRAP_SERVERS": "localhost:9092",
        "KAFKA_CDCTOOL": "True",
        "KAFKA_CONNECTTOOL": "True",
        "KAFKA_CONNECT_URL": "http://localhost:8083",
        "KAFKA_REST_URL": "http://localhost:8082"
      }
    }
  }
}

Alternatively, connect to a pre-deployed Streamable-HTTP instance by url:

{
  "mcpServers": {
    "kafka-mcp": {
      "url": "http://localhost:8000/kafka-mcp/mcp"
    }
  }
}

Run a reviewed container image as a least-privilege stdio child (no listener or published port):

docker run -i --rm \
  --read-only \
  --cap-drop=ALL \
  --security-opt=no-new-privileges \
  --pids-limit=256 \
  --tmpfs /tmp:rw,noexec,nosuid,nodev,size=64m \
  -e TRANSPORT=stdio \
  -e KAFKATOOL=True \
  -e KAFKA_BOOTSTRAP_SERVERS=localhost:9092 \
  -e KAFKA_CDCTOOL=True \
  -e KAFKA_CONNECTTOOL=True \
  -e KAFKA_CONNECT_URL=http://localhost:8083 \
  -e KAFKA_REST_URL=http://localhost:8082 \
  registry.example.invalid/kafka-mcp@sha256:<digest> kafka-mcp

For containerized network HTTP, supply an authenticated TLS ingress (or direct server TLS), exact MCP_ALLOWED_HOSTS, and an exact trusted-proxy CIDR policy through the operator-owned deployment profile. The generator does not emit an unauthenticated non-loopback listener.

Auto-generated from the code-read env surface (package vars) — do not edit.

Additional Deployment Options

kafka-mcp can run as a local stdio process or container, or behind a remote network boundary. The Deployment guide carries the detailed transport contract.

• Local container — launch a reviewed immutable image as a least-privilege stdio child with no listener or published port.
• Remote URL — connect through an operator-supplied authenticated HTTPS ingress. Keep its URL, outbound identity references, trust profile, and exact MCP_ALLOWED_HOSTS in AgentConfig.

Available MCP Tools

Auto-generated — do not edit between the markers below.

Intent tools

13 intent tool(s) · 0 operation(s). Call an operation as act(action='<operation>', params={...}) (ask for reads); find(action='describe', params={'action': '<operation>'}) returns its arguments. An operation is enabled unless its <DOMAIN>TOOL toggle is set false. Auto-generated — do not edit.

Documentation

The complete documentation is published as the official documentation site and is the recommended reference for installation, deployment, and day-to-day operation.

AGENTS.md is the canonical contributor/agent guidance.

Deploy with agent-connector-sdk-deployment

Provision this package with the consolidated agent-connector-sdk-deployment workflow. It selects an installed-package, editable-source, or immutable-container path; records only runtime secret and TLS-profile references in AgentConfig; and runs doctor, registration, policy, observability, and rollback gates. Ask your agent to "deploy kafka-mcp with agent-connector-sdk-deployment".

The repository embeds no deployment profile, credential value, certificate path, or environment-specific endpoint. Supply those at runtime through AgentConfig and the configured secret provider.

Governed capability contract

This package ships a compact canonical skill surface with specialist procedures kept as referenced workflows. The current MCP tools, skill metadata, connector_manifest.yml, ontology, mappings, shapes, fixtures, migrations, tool-schema fingerprints, and certification metadata form one versioned capability contract. Validate them together; do not rely on stale tool names or historical per-task skill wrappers.

Runtime endpoints, credentials, certificate trust, tenant identity, retention, and observability policy are deployment inputs and are never packaged values. See Configuration, trust, and privacy before enabling a network transport, connector ingestion, Arbora delegation, or trace export.