Inference Predictor MCP Server

MCP server for Inference Predictor by Traction Layer AI — predict LLM inference performance (TTFT, throughput, cost) for any Hardware x Model x Runtime configuration, directly from Claude.

Installation

pip install inference-predictor-mcp

Claude Desktop Configuration

Add to your Claude Desktop config (~/Library/Application Support/Claude/claude_desktop_config.json):

{
  "mcpServers": {
    "inference-predictor": {
      "command": "python",
      "args": ["-m", "inference_predictor_mcp"]
    }
  }
}

With a Pro API key (enables compare, optimize, batch-sweep, visualize, SKU listing):

{
  "mcpServers": {
    "inference-predictor": {
      "command": "python",
      "args": ["-m", "inference_predictor_mcp"],
      "env": {
        "KPG_API_KEY": "your-api-key-here"
      }
    }
  }
}

Available Tools

Free Tier (no API key required)

Pro Tier (API key required)

Get a Pro API Key

Visit predictor.tractionlayer.ai to obtain a Pro API key.

Environment Variables

Development

If you're working on both this package and the main KPG_Predictor repo in the same virtualenv, you may hit a starlette version conflict between MCP SDK (requires >=1.0.0) and FastAPI (requires <0.49.0). Resolve by installing MCP first, then explicitly pinning starlette:

pip install -e mcp/
pip install "starlette<0.49.0,>=0.40.0"

End users who only pip install inference-predictor-mcp don't encounter this.

Documentation

Full Inference Predictor documentation: docs/cli.md