Skip to main content

LLMBenchmark.ai MCP server

Ask "which model should I use for this task?" and get an answer from benchmark scores and API prices of 400+ LLMs, updated daily.

  • Free. No API key. 500 requests per IP per UTC day.
  • Remote server, Streamable HTTP: https://api.llmbenchmark.ai/mcp
  • Deterministic: no LLM call inside, the same task and the same data return the same answer.
  • Every answer carries the data date, the source ("LLMBenchmark.ai") and links to the pages behind the numbers.

Docs: https://llmbenchmark.ai/api

Install

Claude Code

claude mcp add --transport http llmbenchmark https://api.llmbenchmark.ai/mcp

Cursor (.cursor/mcp.json)

{
  "mcpServers": {
    "llmbenchmark": { "url": "https://api.llmbenchmark.ai/mcp" }
  }
}

Codex (~/.codex/config.toml)

[mcp_servers.llmbenchmark]
url = "https://api.llmbenchmark.ai/mcp"

Claude Desktop (claude_desktop_config.json, through the mcp-remote bridge)

{
  "mcpServers": {
    "llmbenchmark": {
      "command": "npx",
      "args": ["-y", "mcp-remote", "https://api.llmbenchmark.ai/mcp"]
    }
  }
}

Install as a local server (stdio)

The package proxies to the hosted server, so clients that only start local commands can use it too. Needs uv.

uvx llmbenchmark-mcp

Claude Desktop (claude_desktop_config.json):

{
  "mcpServers": {
    "llmbenchmark": {
      "command": "uvx",
      "args": ["llmbenchmark-mcp"]
    }
  }
}

Docker, built from this repository:

docker build -t llmbenchmark-mcp . && docker run -i --rm llmbenchmark-mcp

Set LLMBENCHMARK_MCP_URL to point the proxy at another endpoint.

Tools

Tool What it does
recommend_model Task text plus optional limits (max_output_price_per_m, min_context_tokens, open_weights_only, top_k) in, ranked models with scores, prices and a one-line reason out.
get_leaderboard Top models on one board: overall, coding, agentic, reasoning, math, knowledge, design.
compare_models Two models side by side on every board where both were measured.
get_model One model: maker, prices, context, open weights, score and rank per board.
get_prices API list prices per million tokens, cheapest first, by model or maker.
list_categories The categories, the board behind each and the task keywords that select it.

recommend_model maps the task to a category with a fixed keyword map (printed at https://llmbenchmark.ai/api#how-recommend-works), drops models that break your limits and ranks the rest on that category's board. If nothing fits your limits it says so and returns the nearest options.

Example prompts

  • "Which model should I use to refactor a React app under $5 per million output tokens?"
  • "What is the best open-weight model for math with at least a 128k context window?"
  • "Compare Claude Opus and GPT-6 Astra for coding."
  • "Which models are cheapest on output price from DeepSeek?"

CLI

llmbenchmark-mcp recommend "refactor a React app" --max-price 5 --open-weights
llmbenchmark-mcp tools
llmbenchmark-mcp config

recommend prints the recommend_model answer, tools lists the tool names, config prints the remote JSON config for Claude or Cursor. With no arguments the command runs the stdio server.

REST

The same answers over plain HTTP:

curl "https://api.llmbenchmark.ai/v1/recommend?task=refactor+a+large+React+codebase&max_output_price_per_m=5"

License

  • This README and the configuration files in this repository: MIT, see LICENSE.
  • The data the server returns: CC BY 4.0. Credit "LLMBenchmark.ai" with a link to https://llmbenchmark.ai. Each score names its source board and the day it was read; the boards keep their own terms.

Run by Take Group.

Metadata

Release files for llmbenchmark-mcp 1.0.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for llmbenchmark-mcp 1.0.0
File Size Uploaded Provenance Actions
llmbenchmark_mcp-1.0.0.tar.gz 6.3 kB Without provenance Details for llmbenchmark_mcp-1.0.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for llmbenchmark-mcp 1.0.0
File Interpreter ABI Platform Actions
llmbenchmark_mcp-1.0.0-py3-none-any.whl Python 3 none any Details for llmbenchmark_mcp-1.0.0-py3-none-any.whl

Total release size: 12.8 kB

Release files / llmbenchmark_mcp-1.0.0.tar.gz

Download URL llmbenchmark_mcp-1.0.0.tar.gz
Size 6.3 kB
Tags Source
SHA-256 checksum
How to use checksums
b41873487aa9bce29faa0a600c575f13ac9cc065bf738efe528889ab527c0ae3
BLAKE2b-256 checksum
How to use checksums
0fc16e1e07dce9b539af57a9e429104565ba42304abecf7cac39dfe9c55e8da7
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.11.29 {"installer":{"name":"uv","version":"0.11.29","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

Release files / llmbenchmark_mcp-1.0.0-py3-none-any.whl

Download URL llmbenchmark_mcp-1.0.0-py3-none-any.whl
Size 6.5 kB
Tags Python 3
SHA-256 checksum
How to use checksums
ac8dc896c8bc7d5c518e5a7128cb5c756b6a38a3d1fdc1889c807e1febaa1bd2
BLAKE2b-256 checksum
How to use checksums
39a0158f626cedf8afc38bcb88017a3a5e987d239e6fce7c29534addac4dcbb6
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.11.29 {"installer":{"name":"uv","version":"0.11.29","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

Security for llmbenchmark-mcp 1.0.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

1.0.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page