forge-mcp 0.1.0

pytest for MCP servers — test any server, no LLM required

🔨 mcp-forge

pytest for MCP servers — test any server, no LLM required.

mcp-forge is a testing framework for Model Context Protocol (MCP) servers. It works with any MCP server — Python, TypeScript, Go, Rust — and requires zero LLM calls. Tests are deterministic, fast, and free to run in CI/CD.

Why?

Most MCP server testing today looks like this: spin up Claude Desktop, type a prompt, squint at the output, and hope it called the right tool. That's vibe-testing.

mcp-forge gives you real tests:

import pytest
from mcp_forge import MCPTestClient

@pytest.fixture
async def client():
    async with MCPTestClient.from_command("python my_server.py") as c:
        yield c

async def test_tool_exists(client):
    tools = await client.list_tools()
    assert "search_files" in [t.name for t in tools]

async def test_search_works(client):
    result = await client.call_tool("search_files", {"query": "README"})
    assert result.is_success
    assert "README.md" in result.text

async def test_search_snapshot(client, forge_snapshot):
    result = await client.call_tool("search_files", {"query": "README"})
    forge_snapshot.assert_match(result)

Installation

pip install forge-mcp

With FastMCP in-memory testing support:

pip install "forge-mcp[fastmcp]"

Quick Start

1. Write tests directly

from mcp_forge import MCPTestClient

async def test_my_server():
    async with MCPTestClient.from_command("python my_server.py") as client:
        # Check tools are exposed
        tools = await client.list_tools()
        assert len(tools) > 0

        # Call a tool
        result = await client.call_tool("greet", {"name": "World"})
        assert result.is_success
        assert "Hello" in result.text

2. Auto-generate tests from a running server

forge init "python my_server.py" -o test_my_server.py

This connects to your server, discovers all tools, and generates a complete test file with example calls for each tool.

3. Record fixtures for snapshot testing

forge record "python my_server.py" -o fixtures/

4. Validate schemas

forge schema "python my_server.py"

Connecting to Servers

STDIO (local servers — most common)

# Python server
async with MCPTestClient.from_command("python my_server.py") as client:
    ...

# TypeScript server
async with MCPTestClient.from_command("npx my-mcp-server") as client:
    ...

# uvx-installed server
async with MCPTestClient.from_command("uvx my-mcp-package") as client:
    ...

# With environment variables
async with MCPTestClient.from_command(
    "python my_server.py",
    env={"API_KEY": "test-key", "DATABASE_URL": "sqlite:///test.db"},
) as client:
    ...

FastMCP in-memory (zero overhead)

from my_server import mcp  # your FastMCP instance

async with MCPTestClient.from_fastmcp(mcp) as client:
    ...

Features

Tool Testing

async def test_tool_call(client):
    result = await client.call_tool("calculate", {"expression": "2 + 2"})
    assert result.is_success
    assert result.text == "4"

async def test_tool_error_handling(client):
    result = await client.call_tool("calculate", {"expression": ""})
    assert result.is_error

async def test_tool_timeout(client):
    # Tool calls have configurable timeouts
    result = await client.call_tool("slow_operation", timeout=5.0)

Snapshot Testing

async def test_snapshot(client, forge_snapshot):
    result = await client.call_tool("get_user", {"id": "123"})

    # First run: saves snapshot to __snapshots__/
    # Next runs: compares against saved snapshot
    forge_snapshot.assert_match(result)

    # Ignore volatile fields
    forge_snapshot.assert_match(result, ignore_keys=["timestamp", "request_id"])

async def test_schema_snapshot(client, forge_snapshot):
    tools = await client.list_tools()
    # Detect when tool schemas change unexpectedly
    forge_snapshot.assert_schema_unchanged(tools)

Update snapshots:

pytest --update-snapshots

Schema Validation

from mcp_forge import assert_tool_schema, assert_result_matches_schema

async def test_tool_has_correct_schema(client):
    tools = await client.list_tools()
    search_tool = next(t for t in tools if t.name == "search")

    assert_tool_schema(search_tool, {
        "type": "object",
        "required": ["query"],
        "properties": {
            "query": {"type": "string"},
            "limit": {"type": "integer"},
        },
    })

Resource Testing

async def test_resources(client):
    resources = await client.list_resources()
    assert len(resources) > 0

    content = await client.read_resource("file:///README.md")
    assert content.is_success

Prompt Testing

async def test_prompts(client):
    prompts = await client.list_prompts()
    assert "summarize" in [p.name for p in prompts]

    prompt = await client.get_prompt("summarize", {"text": "Hello world"})
    assert prompt.is_success

CLI Reference

forge init <command>       Generate test file from a running server
forge record <command>     Record tool responses as test fixtures
forge schema <command>     Dump tool schemas (JSON/YAML/table)
forge check <command>      Run built-in health checks

Pytest Fixtures

mcp-forge auto-registers these fixtures via the pytest plugin:

Fixture	Description
forge_snapshot	Snapshot manager for the current test
forge_server	Factory for creating MCPTestClient instances

Comparison with Existing Tools

Feature	mcp-forge	FastMCP testing	pytest-mcp	mcp-validator
Any MCP server	✅	❌ FastMCP only	✅	✅
No LLM required	✅	✅	❌	Partial
pytest-native	✅	Manual	✅	❌ CLI only
Snapshot testing	✅	❌	❌	❌
Auto-gen fixtures	✅	❌	❌	❌
Schema validation	✅	❌	❌	✅

Roadmap

• v0.1 (current): pytest plugin, STDIO transport, snapshots, schema validation
• v0.2: Fuzz testing, contract testing, HTTP/SSE transport, GitHub Action
• v0.3: VSCode extension, performance testing, coverage reports

Contributing

Contributions welcome! See CONTRIBUTING.md for guidelines.

License

MIT