ShipCheck

It doesn't write tests. It proves the feature works.

In a nutshell

AI coding agents (Claude Code, Cursor, OpenCode…) can write code and write tests, but nothing checks the agent's own work the way a QA engineer would: actually run the app, click through the real flow, hit the real API, and look at the real database.

ShipCheck is that QA engineer, exposed as an MCP server. Point it at any repository and describe a feature — it discovers how the app actually works (no hardcoded assumptions about routes, tables, or auth patterns), plans what to check, and verifies it across the UI, the API, and the database, handing back evidence instead of a guess.

ANY SOFTWARE
     │
     ▼
 INSPECT · DISCOVER          detect stack, build a SystemGraph of routes/tables/actors
     │
     ▼
    PLAN                     generate a verification plan from the graph — no hardcoded URLs
     │
     ▼
   VERIFY                    Playwright (UI) + HTTP (API) + read-only SQL (database)
     │
     ▼
   EVIDENCE                  pass/fail per layer, diagnosis on failure, a written report

Install

Published on PyPI as shipcheck-mcp (the import name is still shipcheck, the MCP server and CLI are the same package — mcp is a base dependency, not an extra, so the plain command below is enough to run the MCP server):

pip install shipcheck-mcp                              # MCP server + CLI, no extras needed
pip install 'shipcheck-mcp[playwright,database]'        # + browser-driven UI checks + SQL verification
playwright install chromium                             # only if you installed the playwright extra

From a clone of this repo instead: pip install -e '.[playwright,database]'.

Use it from an agent

Zero local install — an MCP client can run it directly via uvx:

claude mcp add shipcheck -- uvx shipcheck-mcp

Want UI/database checks too:

claude mcp add shipcheck -- uvx --from 'shipcheck-mcp[playwright,database]' shipcheck-mcp

Then just ask:

"Use ShipCheck to verify that users can register and log in."

The agent calls ShipCheck's tools directly — discover, verify_feature, bughunt, verify_change — the same way it calls a file-read or shell tool.

Use it from the terminal

shipcheck inspect .                                          # detect the stack, no config file needed
shipcheck plan "users can register and log in"                # see the plan, dry-run, nothing executed
shipcheck verify "users can register and log in" \
  --execute --base-url http://localhost:8000                  # run it for real
shipcheck loop "users can register and log in" \
  --execute --base-url http://localhost:8000 --depth advanced # verify, diagnose failures, retest

What it verifies

Beyond the happy path: autonomous bug hunting (composes every operation workflow from the graph and finds what's broken, no feature description needed), bug filing to GitHub issues, discovery-driven test generation (negative/boundary/auth/state/regression), behavioral invariants (idempotency, state conservation, auth boundaries), git-diff-scoped verification (verify_change — only re-checks what a diff actually affects), and CI reporting (markdown/HTML/JUnit + GitHub PR comments).

27 MCP tools

Full reference: website/pages/docs/tools.mdx (run cd website && npm install && npm run dev to browse it, or read AGENTS.md). Grouped by what they do: discovery (inspect_project, discover, explore, capabilities), planning & verification (generate_test_plan, verify_feature, run_verification_loop_tool, analyze_diff, verify_change), autonomous (bughunt, report_bugs, multi_actor_verify), API & data (api_request, api_discover, api_set_token, db_query, db_discover, db_assert), record & report (save_scenario, load_scenario, list_scenarios, generate_report, github_pr_report), memory (project_memory, show_memory_graph, record_memory_change, observe).

Design goal

ShipCheck doesn't compete with Playwright MCP or browser-use as a low-level browser controller. Those answer how do I interact with a page? ShipCheck answers what should be verified, which users need to participate, what state should change, and what evidence proves the feature works?

It doesn't know your application. It discovers it, learns how it behaves, and proves whether the behavior you asked for actually works — the same way on a FastAPI JSON API, a Next.js SPA, or a classic server-rendered Flask app, without a line of product-specific code in ShipCheck itself.

Docs

• Getting Started · Architecture · Deployment · Roadmap
• Docs site: cd website && npm install && npm run dev

License

MIT