mcp-trust-check

One GitHub Action that runs the full MCP trust & safety trilogy — mcp-doctor, mcp-fuzz, mcp-reality-check — against an MCP server and posts one combined score, instead of three separate installs and three separate reports. Also a Python package (pip install mcp-trust-check) doing the same job for a live agent session instead of a CI run — see Live session wrapper below.

Each of the three tools already works standalone and does something the other two structurally can't:

• mcp-doctor reads the server's source and checks spec conformance/security/documentation — no server execution needed.
• mcp-fuzz actually launches the server and calls every tool with schema-derived inputs, checking crash resilience, latency, response size, and (opt-in) concurrency and resource-lifecycle correctness.
• mcp-reality-check also launches the server, but checks whether a tool's successful response is actually trustworthy — no disguised refusals, no empty content, no output that quietly ignores its own schema.

None of them use an LLM judge anywhere — every check here is deterministic and reproducible, same as the tools it wraps.

Demo

35 seconds, live run against the official MCP reference server — real output, no cherry-picking: 86%/B code quality, 100%/A crash resilience, 100%/A output fidelity, 95%/A combined.

Use

Static-only — zero extra config, works on any checked-out repo:

- uses: actions/checkout@v4
- uses: vishalhabib99/mcp-trust-check@v1
  with:
    path: .

Full trilogy — also add run with the command that actually starts your server over stdio:

- uses: actions/checkout@v4
- uses: vishalhabib99/mcp-trust-check@v1
  with:
    path: .
    run: "python server.py"
    # or: run: "npx -y some-mcp-server"
    fail-under: 80

If run is left empty, only the mcp-doctor static check runs — the runtime checks are skipped and the report says so explicitly, not silently dropped. This is a real tradeoff, not a default to route around: mcp-doctor can statically analyze any checked-out repo path with no configuration, but mcp-fuzz and mcp-reality-check have to actually launch your server, and there's no way to derive a correct launch command from a repo checkout in general — you have to supply it.

Inputs

Outputs

doctor-score / doctor-grade, fuzz-score / fuzz-grade, reality-score / reality-grade (empty if run wasn't set), and combined-score / combined-grade.

What the combined score means — and doesn't

The combined score is a plain, unweighted average of whichever of the three scores actually ran (one or three — never two, since fuzz and reality-check both need run or neither runs). No tool is weighted more heavily than another; that would require a judgment call about which failure mode matters more that this project isn't going to make for you. Treat it as a single skim-friendly number for a PR check, not a substitute for reading the three sections underneath it — each retains its own real caveats (mcp-fuzz's crash-resilience score, for instance, deliberately doesn't grade whether a successful call's output was actually correct; that's what the reality-check section is for).

This repo contains no new detection logic of its own — it's orchestration over the three published tools, each independently dogfooded against 40+ real-world MCP servers (see each tool's own README for that history). If a check here is wrong, the bug is almost certainly in the underlying tool, not in the combining step.

Live session wrapper

The Action above runs once, in CI. By now, each tool in the trilogy also has its own live counterpart, usable directly in an agent's own code: mcp-doctor's registration gate, mcp-fuzz's latency gate, mcp-reality-check's correctness gate. GuardedSession is the same idea as this Action, applied there: one wrapper around a real ClientSession that runs all three at their natural point, instead of three separate imports wired by hand.

from mcp import ClientSession
from mcp_trust_check import GuardedSession

session: ClientSession = ...  # your own, already-connected session
gs = GuardedSession(session)

await gs.list_tools()  # runs mcp-doctor's checks once, at discovery
for name, result in gs.registration_results.items():
    if result.flagged:
        ...  # missing/vague description, or a real annotation conflict

result = await gs.call_tool("some_tool", {"arg": "value"})
if result.outcome != "ok":
    ...  # crashed or timed out
elif result.flagged:
    ...  # slow/bloated relative to this tool's own history, a disguised
         # refusal, empty content, or a schema violation

One real call per call_tool, not three — the reason this exists as its own composed wrapper rather than "just call all three gates yourself": mcp_fuzz.gate.LatencyGate.timed_call and mcp_reality_check.gate.guarded_call each make their own real call to the tool. Calling both back to back would mean two real invocations per logical call — wasteful for an idempotent tool, actively wrong for a non-idempotent or destructive one. GuardedSession calls the tool exactly once and fans the single real response out to each sibling package's own pure, already-tested per-response functions instead — verified directly: a real test wraps the underlying session's call_tool with a call counter and asserts it fires exactly once per GuardedSession.call_tool.

Same scope discipline as its three parts: correctness and latency, not security — see each sibling's own README for why that's a deliberate boundary, not an oversight. Dogfooded live against the official @modelcontextprotocol/server-memory reference server — registration, a real create_entities call, and a real read_graph call, all clean.

License

MIT