mcpact 0.2.0

Contract testing and regression detection for MCP servers.

mcpact

Contract testing for MCP servers.

You build an MCP server. You change something. How do you know the tools still work and the schemas haven't drifted? mcpact gives you a YAML contract, runs it against your running server, and fails CI when something breaks.

Install

pip install mcpact

Requires Python 3.10+.

A minimal contract

# contracts/my-server.yaml
server:
  transport: stdio
  command: python server.py

tools:
  - name: search_files
    description_contains: search
    input_schema:
      required: [query]
      properties:
        query: { type: string }
    assertions:
      - call:
          args: { query: "hello" }
        expect:
          status: success
          max_latency_ms: 1000

Run it:

mcpact run --contract contracts/my-server.yaml

search_files
  ✓ exists
  ✓ description_contains
  ✓ input_schema.required
  ✓ call#1.status [42ms]
  ✓ call#1.max_latency_ms [42ms]

5 checks · 5 passed · 0 failed

Exit code is 0 on success, 1 on any failure, 2 on contract errors.

Detect regressions

Capture the current surface, then compare future runs against it:

mcpact snapshot --contract contracts/my-server.yaml --out .mcpact/baseline.json
mcpact diff     --contract contracts/my-server.yaml --baseline .mcpact/baseline.json

Snapshot diff:
  + get_file_info          (new tool)
  - delete_file            (removed — breaking)
  ✗ search_files
    └─ required removed: limit
    └─ type changed on query: 'string' → 'integer'

diff exits non-zero on breaking changes (removed tools, removed required fields, changed property types) so you can wire it into CI.

Transports

server:
  transport: stdio     # command: python server.py
  transport: http      # url: http://localhost:3000/mcp
  transport: sse       # url: http://localhost:3000/sse

Commands

Command	Purpose
mcpact run	Run a contract against a server
mcpact watch	Re-run the contract on file changes
mcpact snapshot	Write the server's tool surface to a JSON file
mcpact diff	Compare a live server against a snapshot
mcpact validate	Validate a contract file without running it

Reporters for run: console (default), json, junit, html. Output goes to --out when provided, otherwise to stdout.

For a realistic example, see packages/core/examples/notes_server.py and the matching contract contracts/notes.yaml — a tiny notes MCP server exercising list, read, search, and create tools.

GitHub Actions

- run: pip install mcpact
- run: mcpact run --contract contracts/my-server.yaml --reporter junit --out results.xml
- uses: mikepenz/action-junit-report@v4
  if: always()
  with:
    report_paths: results.xml

Python API

import asyncio
from mcpact import build_client, load_contract, run_contract

async def main() -> None:
    contract = load_contract("contracts/my-server.yaml")
    client = build_client(contract.server)
    await client.connect()
    try:
        report = await run_contract(contract, client)
    finally:
        await client.close()
    print(f"{report.passed}/{report.total} passed")

asyncio.run(main())

Contract reference

server:
  transport: stdio | http | sse
  command: <cmd>           # stdio only
  args: [<arg>, ...]       # stdio only
  env: { KEY: value }      # stdio only
  url: <url>               # http/sse only

before:                    # hooks: shell or tool call (contract-level)
  - shell: "setup.sh"
  - tool: seed_db
    args: { count: 10 }
after:
  - shell: "cleanup.sh"

tools:
  - name: <tool>
    must_exist: true       # default
    description_contains: <str> | [<str>, ...]
    input_schema:
      required: [<field>, ...]
      properties:
        <field>: { type: string|integer|... }
    before: [...]          # per-tool hooks
    after: [...]
    assertions:
      - name: <step_name>  # optional; enables variable references
        call:
          args: { ... }
          timeout_ms: <int>
        expect:
          status: success | error    # default: success
          response_contains: <str> | [<str>, ...]
          max_latency_ms: <int>
          schema: { ... }            # JSON Schema for the response
      - call:
          args:
            id: "${{ steps.<step_name>.result.id }}"   # variable from named step
        expect: { status: success }

resources:
  - uri: "config://version"
    content_contains: "version"
    content_schema: { type: object, required: [version] }
  - uri_pattern: "file:///data/.*"
    min_count: 1

prompts:
  - name: summarize
    description_contains: "Summarize"
    arguments:
      - name: text
        required: true
    assertions:
      - get_prompt:
          args: { text: "Hello" }
        expect:
          message_count: 1
          messages_contain: "summary"

Status

Pre-1.0. Both packages are functional and tested against a real MCP server. API may still change before 1.0.

v0.2 adds: resources and prompts testing, named assertions with variable references (${{ steps.X.result.path }}), before/after hooks (shell + tool calls), --no-hooks flag.

Repository layout

packages/core/        Python engine + CLI
packages/sdk-node/    TypeScript SDK
contracts/            Example contracts

Contributing

See CONTRIBUTING.md. Issues and PRs welcome — please open an issue before starting non-trivial work. Security issues: see SECURITY.md.

License

MIT