mcp-stdio-test 0.1.0

Zero LLM, zero network, zero API keys - a CI-gate harness for stdio MCPs.

mcp-stdio-test

Zero LLM, zero network, zero API keys - a CI-gate harness for stdio MCPs.

mcp-stdio-test spawns an MCP server over stdio, runs the JSON-RPC initialize handshake, and lets you assert on tools/list or call any tool - all with CI-friendly exit codes. It is the thing you wire into .github/workflows/ci.yml so a missing tool or a broken signature turns the pipeline red without anyone reading the output.

If you want a scored quality report, use mcp-doctor. If you want rule-based lint, use mcp-lint. If you want agent-driven end-to-end evaluation with cost tracking, use mcp-tester. If you want to click through your server, use MCP Inspector. mcp-stdio-test is the harness.

Install

pip install mcp-stdio-test
# or
pipx install mcp-stdio-test
# or (no install)
uvx mcp-stdio-test --help

Zero runtime dependencies. Python 3.10+.

60-second quickstart

# 1. Confirm the tool works on your machine.
mcp-stdio-test doctor

# 2. Point it at any stdio MCP server script.
mcp-stdio-test path/to/server.py --list-tools

# 3. Gate CI on the tool inventory.
mcp-stdio-test path/to/server.py --list-tools --expect-count 5 --expect-tool search

A non-zero exit code means something regressed. No log-scraping required.

CLI reference

Flag	Purpose
<server>	Path to the MCP server script to spawn.
--pyexe PATH	Python interpreter to run the server with (default: sys.executable).
--list-tools	Print the tool inventory returned by tools/list.
--call TOOL	Invoke a single tool by name.
--args JSON	JSON-encoded arguments for --call (default: {}).
--expect-count N	Assert tools/list returns exactly N tools. Exit 4 on mismatch.
--expect-tool NAME	Assert a tool with NAME is registered. Repeatable. Exit 4 on miss.
--timeout SECONDS	Wall-clock deadline (default 30). Kills hung servers.
--json	Emit machine-readable JSON instead of text.
doctor	Self-check: Python version, optional mcp SDK, stdio handshake.

Exit codes

Code	Meaning
0	OK
1	initialize / handshake failed
2	tools/list failed
3	tools/call failed
4	assertion failed (--expect-count / --expect-tool mismatch)
5	usage error (bad flags, missing server path)

Examples

Fresh-MCP smoke test

mcp-stdio-test my_mcp/server.py --list-tools

Regression gate when adding a tool

# Before the change: 4 tools.
# After the change: assert 5.
mcp-stdio-test my_mcp/server.py --list-tools --expect-count 5

CI integration (GitHub Actions)

- run: pip install mcp-stdio-test
- run: mcp-stdio-test my_mcp/server.py --list-tools --expect-count 5 --expect-tool search

Pre-commit hook

repos:
  - repo: https://github.com/dtchen07/mcp-stdio-test
    rev: v0.1.0
    hooks:
      - id: mcp-stdio-test

Comparison with neighbors

Tool	Primary job	Transport	LLM	Exit-code CI	Tool-count / tool-name assertions
mcp-stdio-test (this)	Per-repo declarative CI gates	stdio	no	yes (0/1/2/3/4 tiered)	yes (first-class, user-authored)
mcp-lint (LuxshanLux)	23 deterministic lint rules, scored	stdio + SSE	no	yes (0/1/2)	no
mcp-tester (saqadri, "MCP-Eval")	Pytest-style agent evaluation + cost	stdio	yes	JSON + pytest	no
mcp-doctor (destilabs)	Diagnostic + agent-friendliness scorer	stdio + HTTP	partial (generate-dataset only)	scored report	no
mcp-probe (conikeec)	Rust TUI debugger + compliance suite	stdio / SSE / HTTP	no	--fail-fast pass/fail	built-in suite, not user-authored
MCP Inspector (official)	Interactive debugger + --cli	stdio / SSE / HTTP	no	JSON (no assertions)	no

Two distinctions that hold up on direct comparison: none of the five ship per-repo, user-authored tool-count / tool-name assertions, and only two publish numbered CI exit codes at all (mcp-lint's 0/1/2, mcp-probe's pass/fail) — neither tiered by handshake vs tool-count vs usage like the 0/1/2/3/4 ladder here.

Compatibility

This release	Tested against
0.1.0	MCP protocol 2024-11-05; mcp SDK >=1.2,<2 for optional user fixtures

Support matrix: Python 3.10, 3.11, 3.12, 3.13 on Linux, macOS, Windows.

Non-goals

• Not a framework. Do not wrap your server in it.
• Not a pytest plugin. Use subprocess from pytest if you want; no plugin glue.
• No HTTP / SSE / streaming transport - stdio only.
• No LLM-in-the-loop evaluation.
• No multi-MCP orchestration.

The why

The long-form rationale, evidence from 6 weeks of real usage, and a walkthrough from empty repo to green CI is in the launch blog post: https://dtchen07.github.io/2026/04/harness-first-mcp/

Maintenance

This is a personal tool released as-is. Issue response target: 7 days, best-effort; no SLA on fixes. PRs are merged if they ship with tests and stay within scope; feature requests outside the non-goals will be declined politely. See RELEASE.md for the release runbook and the launch blog post for kill-criteria.

A field-data-backed 0.2.0 is targeted ~8 weeks after 0.1.0 ships.

Verify the supply chain

Every release is published via PyPI Trusted Publishing and ships with PEP 740 build provenance. Verify from your shell:

pip download mcp-stdio-test
gh attestation verify mcp_stdio_test-*.whl --owner dtchen07

License and citation

Code: MIT. Citation metadata in CITATION.cff. Contributing guide: CHANGELOG.md plus the PR template.