Skip to main content

toolrate-mcp

Official Python Model Context Protocol server for ToolRate — the reliability oracle for AI agents.

Python sibling of @toolrate/mcp-server. Same 9 tools, same guard pattern, different runtime — pick whichever your editor's MCP integration prefers.

  • toolrate_assess — reliability score, pitfalls, mitigations, and best alternatives for any third-party API.
  • toolrate_route_llm — picks the right LLM model+provider under your token budget across 7 providers (Anthropic, OpenAI, Groq, Together, Mistral, DeepSeek, Ollama).
  • toolrate_report — feed real outcomes back so scores stay accurate.
  • toolrate_fallback_chain — what real agents switch to when a tool fails.
  • toolrate_hidden_gems — underrated tools that excel as fallbacks.
  • toolrate_search_tools, toolrate_categories, toolrate_my_usage — browse the catalog and watch your quota.

Setup

You need a free ToolRate API key — get one at toolrate.ai, or call the bootstrap toolrate_register tool from any MCP client (only available when no key is configured yet).

The recommended runner is uv/uvx, which fetches and runs the package on demand with no permanent install — same story as npx for the TypeScript sibling.

Claude Code

claude mcp add toolrate --env TOOLRATE_API_KEY=nf_live_xxxxx -- uvx toolrate-mcp

Cursor — ~/.cursor/mcp.json

{
  "mcpServers": {
    "toolrate": {
      "command": "uvx",
      "args": ["toolrate-mcp"],
      "env": {
        "TOOLRATE_API_KEY": "nf_live_xxxxx"
      }
    }
  }
}

Zed — ~/.config/zed/settings.json

{
  "context_servers": {
    "toolrate": {
      "command": {
        "path": "uvx",
        "args": ["toolrate-mcp"],
        "env": { "TOOLRATE_API_KEY": "nf_live_xxxxx" }
      }
    }
  }
}

Other MCP clients

Anything that speaks MCP over stdio works the same way — point it at uvx toolrate-mcp (or python -m toolrate_mcp after a plain pip install toolrate-mcp) with TOOLRATE_API_KEY in env.

Tools

Tool What it does
toolrate_assess Rate a tool's reliability before calling. Returns a summary block + 20 detail fields.
toolrate_report Report success/failure after a tool call.
toolrate_route_llm Pick the best LLM model+provider under your budget.
toolrate_fallback_chain Get the best alternatives when a tool fails.
toolrate_hidden_gems Discover underrated tools by category.
toolrate_search_tools Search/browse the ToolRate catalog.
toolrate_categories List all canonical categories.
toolrate_my_usage Check your API key quota.
toolrate_register (Only when TOOLRATE_API_KEY is unset) Register a new key by email.

How agents should use this

Drop this into your agent's system prompt — see prompts/guard-pattern.md for the canonical version:

Before calling any third-party API tool, first call toolrate_assess with the tool's identifier (URL or canonical name). Read summary.reliability_score.

  • If score ≥ 70, proceed.
  • If score < 70 and top_alternatives has a fit, use the highest-scoring alternative.
  • If score < 40, surface the risk to the user before proceeding.

After every third-party API tool call, call toolrate_report with {tool_identifier, success, error_category?, latency_ms?}. This is how ToolRate learns.

When picking an LLM provider for a generation task, prefer toolrate_route_llm — it returns the routing fields directly.

Quota tips

  • Free tier is 100 assessments/day. ToolRate caches scores for ~5 min server-side so calling assess for the same tool twice in a row only costs one quota call.
  • Run toolrate_route_llm once per workflow and reuse the picked model — it costs 1 quota call per provider considered (up to 7).
  • Use toolrate_my_usage to self-throttle.

Environment variables

Variable Required Default Purpose
TOOLRATE_API_KEY Yes (except for toolrate_register) — Your nf_live_… API key.
TOOLRATE_BASE_URL No https://api.toolrate.ai Override for staging/dev.

Development

git clone https://github.com/netvistamedia/toolrate
cd toolrate/mcp/python

python -m venv .venv
source .venv/bin/activate
pip install -e .

# Smoke test (verifies tool registration in both with-key and no-key modes)
python smoke.py

# Wire into Claude Code locally
claude mcp add toolrate-dev --env TOOLRATE_API_KEY=nf_live_xxxxx -- python -m toolrate_mcp

License

MIT — distinct from the ToolRate API server (BUSL-1.1). The MCP server is permissively licensed so any project can integrate it.

Metadata

Release files for toolrate-mcp 0.1.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for toolrate-mcp 0.1.1
File Size Uploaded Provenance Actions
toolrate_mcp-0.1.1.tar.gz 14.3 kB Without provenance Details for toolrate_mcp-0.1.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for toolrate-mcp 0.1.1
File Interpreter ABI Platform Actions
toolrate_mcp-0.1.1-py3-none-any.whl Python 3 none any Details for toolrate_mcp-0.1.1-py3-none-any.whl

Total release size: 35.7 kB

Release files / toolrate_mcp-0.1.1.tar.gz

Download URL toolrate_mcp-0.1.1.tar.gz
Size 14.3 kB
Tags Source
SHA-256 checksum
How to use checksums
23e9b335ea07a3f2794ff07aa7f6a58731d7eae745a6a1ea34e29efbdd80c422
BLAKE2b-256 checksum
How to use checksums
5a8c4d952bba5da525902b093701396580b1f6e7a838c42c7ce5cab591744084
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.14.3

Release files / toolrate_mcp-0.1.1-py3-none-any.whl

Download URL toolrate_mcp-0.1.1-py3-none-any.whl
Size 21.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
a37664351073a725f458d8fd5bc8260860eb4289a08454d1193fb5fceeb53d92
BLAKE2b-256 checksum
How to use checksums
8c2a8a2b3fcf67c5c94f0830f58a03ec399bff4fd4e4282c564aeae47d365a49
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.14.3

Security for toolrate-mcp 0.1.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.1 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page