inference-predictor-mcp 0.1.0

MCP server for LLM inference performance prediction — Traction Layer AI

Inference Predictor MCP Server

MCP server for Inference Predictor by Traction Layer AI — predict LLM inference performance (TTFT, throughput, cost) for any Hardware x Model x Runtime configuration, directly from Claude.

Installation

pip install inference-predictor-mcp

Claude Desktop Configuration

Add to your Claude Desktop config (~/Library/Application Support/Claude/claude_desktop_config.json):

{
  "mcpServers": {
    "inference-predictor": {
      "command": "python",
      "args": ["-m", "inference_predictor_mcp"]
    }
  }
}

With a Pro API key (enables compare, optimize, batch-sweep, visualize, SKU listing):

{
  "mcpServers": {
    "inference-predictor": {
      "command": "python",
      "args": ["-m", "inference_predictor_mcp"],
      "env": {
        "KPG_API_KEY": "your-api-key-here"
      }
    }
  }
}

Available Tools

Free Tier (no API key required)

Tool	Description
predict_performance	Predict TTFT, throughput, cost for a single hardware config
check_hardware_compatibility	Check which GPUs can fit a model's weights
explain_model	Generate educational architecture explainer
list_models	List all 18 registered models with parameters
health_check	Check API health and version

Pro Tier (API key required)

Tool	Description
compare_configs	Compare vLLM, SGLang, TensorRT-LLM side-by-side
find_optimal_hardware	Search for cheapest/fastest hardware config
batch_size_sweep	Sweep batch sizes to find optimal throughput
visualize_kpg	Generate interactive Kernel Pipeline Graph
list_hardware_skus	List AWS GPU instance SKUs with pricing

Get a Pro API Key

Visit predictor.tractionlayer.ai to obtain a Pro API key.

Environment Variables

Variable	Default	Description
KPG_API_BASE_URL	Production API Gateway	Override for self-hosted deployments
KPG_API_KEY	(none)	Pro tier API key
KPG_TIMEOUT	30	HTTP timeout in seconds

Development

If you're working on both this package and the main KPG_Predictor repo in the same virtualenv, you may hit a starlette version conflict between MCP SDK (requires >=1.0.0) and FastAPI (requires <0.49.0). Resolve by installing MCP first, then explicitly pinning starlette:

pip install -e mcp/
pip install "starlette<0.49.0,>=0.40.0"

End users who only pip install inference-predictor-mcp don't encounter this.

Documentation

Full Inference Predictor documentation: docs/cli.md