testio-mcp 0.1.1

Model Context Protocol server for TestIO Customer API integration

TestIO MCP Server

AI-first Model Context Protocol server for TestIO Customer API

Query TestIO test status, bugs, and activity metrics through AI tools like Claude and Cursor—no UI required.

⚠️ Disclaimer: This software is provided "AS IS" with no warranty or support obligations. See License & Disclaimer for details.

---

What Is This?

TestIO MCP Server is a Model Context Protocol (MCP) server that provides read-only access to the TestIO Customer API. It enables non-developer stakeholders (CSMs, PMs, QA leads) to query test information using natural language through AI assistants like Claude Desktop or Cursor.

Key Features:

• 🔍 9 MCP Tools - Health check, test status, bug filtering, activity analysis, status reports, cache monitoring
• 🏗️ Service Layer Architecture - Framework-agnostic business logic, reusable across transports
• 🔒 Security-First - Token sanitization, strict input validation, comprehensive secret scanning
• ⚡ Performance - Smart caching (1h/5m/1m TTL), connection pooling, concurrency control
• 📊 Strict Type Safety - mypy --strict enforced, Pydantic v2 validation throughout

Use Cases:

• CSMs: "What's the status of test 109363?" → Get comprehensive test details in seconds
• TLs: "Show me critical functional bugs for test 109363" → Filter bugs by type, severity, status
• CSMs: "Generate a status report for tests 109363, 109364" → Export markdown/text/json summaries

---

Quick Start

Prerequisites

• Python 3.12 or higher
• uv package manager (recommended)
• TestIO Customer API token (staging or production)
• Claude Desktop or Cursor (AI client with MCP support)

Installation

# 1. Clone the repository
git clone https://github.com/test-IO/customer-mcp.git
cd customer-mcp

# 2. Create virtual environment and install dependencies
uv venv
uv pip install -e ".[dev]"

# 3. Install pre-commit hooks (optional but recommended)
pre-commit install

# 4. Configure API credentials
cp .env.example .env
# Edit .env and set TESTIO_CUSTOMER_API_TOKEN=your-token-here

# 5. Verify installation
uv run pytest -m unit  # Run fast unit tests (no API needed)
uv run python -m testio_mcp --help  # Show CLI help

CLI Usage

The testio-mcp command supports various flags for configuration and troubleshooting:

# Show help
testio-mcp --help

# Show version
testio-mcp --version

# Use custom .env file for credentials
testio-mcp --env-file ~/testio-prod.env

# Override cache settings
testio-mcp --cache-ttl-products 7200 --cache-ttl-tests 600

# Enable debug logging
testio-mcp --log-level DEBUG --log-format text

# Combine multiple options
testio-mcp --env-file ~/testio-staging.env --log-level DEBUG --cache-ttl-bugs 120

Available CLI Flags:

Flag	Description	Default
--help, -h	Show help message and exit	-
--version, -v	Show version and exit	-
--env-file PATH	Load credentials from custom .env file	.env in current dir
--cache-ttl-products SECONDS	Product cache TTL	3600 (1 hour)
--cache-ttl-tests SECONDS	Test data cache TTL	300 (5 minutes)
--cache-ttl-bugs SECONDS	Bug data cache TTL	60 (1 minute)
--log-level LEVEL	Logging level (DEBUG, INFO, WARNING, ERROR, CRITICAL)	INFO
--log-format FORMAT	Log format (json, text)	json
--max-concurrent-requests N	Max concurrent API requests (1-50)	10
--connection-pool-size N	HTTP connection pool size (1-100)	20
--http-timeout SECONDS	HTTP timeout (1-300)	30.0

Credentials:

• API token and base URL must be provided via environment variables or --env-file
• Never pass credentials as CLI arguments (security risk)
• Use .env files for different environments (prod, staging, dev)

Graceful Shutdown:

• Press CTRL+C to stop the server gracefully
• Server handles SIGINT and SIGTERM signals for clean exit

MCP Client Configuration

All MCP clients use the same basic configuration format:

{
  "mcpServers": {
    "testio-mcp": {
      "command": "/absolute/path/to/customer-mcp/.venv/bin/python",
      "args": ["-m", "testio_mcp"]
    }
  }
}

Supported Clients:

• Claude Code (CLI) - ~/.config/claude/code_config.json
• Claude Desktop - macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
• Cursor - ~/.cursor/mcp.json or .cursor/mcp.json
• Gemini - CLI: gemini mcp add testio-mcp /path/to/python -m testio_mcp
• Codex - ~/.codex/config.toml (TOML syntax)

📖 For detailed setup instructions by client, see MCP_SETUP.md

First Query

Restart Claude Desktop or Cursor, then try:

What's the status of test 109363?

You should see comprehensive test details including title, status, timeline, and bug summary.

---

Features & API Reference

Available Tools

Tool	Description	Key Parameters	Example Query
health_check	Verify API authentication	None	"Check TestIO API health"
get_test_status	Get comprehensive test status	test_id (int)	"What's the status of test 109363?"
list_tests	List tests for a product	product_id (int), statuses (optional), include_bug_counts (bool)	"Show me all active tests for product 25073"
list_products	List available products	search (optional), product_type (optional)	"List all products" or "Find products with 'mobile' in name"
get_test_bugs	Filter bugs by type/severity/status	test_id (str), bug_type, severity, status, page_size, continuation_token	"Show me critical functional bugs for test 109363"
generate_status_report	Generate executive summaries	test_ids (list[str]), format (markdown/text/json)	"Generate a status report for tests 109363, 109364 in markdown"
get_test_activity_by_timeframe	Analyze activity across products	product_ids (list[str]), start_date, end_date, date_field, include_bugs	"Show test activity for product 25073 from 2024-10-01 to 2024-12-31"
get_cache_stats	Monitor cache performance metrics	None	"Show me cache statistics" or "What's the cache hit rate?"
clear_cache	Clear all cached data (admin tool)	None	"Clear the cache" or "Force refresh all data"

Performance Characteristics

• Caching: Smart TTL-based caching reduces API load
  • Products: 1 hour
  • Tests: 5 minutes
  • Bugs: 1 minute
• Concurrency: Max 10 concurrent API requests (configurable)
• Response Times: <5 seconds for 99% of queries
• Connection Pooling: Max 100 connections, 20 keep-alive

---

Architecture Overview

TestIO MCP Server follows a service layer architecture that separates business logic from transport mechanisms:

┌─────────────────────────────────────────────────────────────┐
│                     MCP Tools (Thin Wrappers)               │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐        │
│  │health_   │ │get_test_ │ │list_     │ │get_test_ │  ...   │
│  │check     │ │status    │ │tests     │ │bugs      │        │
│  └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘        │
│       │            │            │            │              │
│       └────────────┴────────────┴────────────┘              │
│                          │                                  │
│            Extract deps, delegate, convert errors           │
│                          ▼                                  │
└─────────────────────────────────────────────────────────────┘
                           │
┌─────────────────────────────────────────────────────────────┐
│              Service Layer (Business Logic)                 │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐        │
│  │Test      │ │Bug       │ │Product   │ │Activity  │  ...   │
│  │Service   │ │Service   │ │Service   │ │Service   │        │
│  └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘        │
│       │            │            │            │              │
│       └────────────┴────────────┴────────────┘              │
│                          │                                  │
│       Domain operations, caching, orchestration             │
│                          ▼                                  │
└─────────────────────────────────────────────────────────────┘
                           │
┌────────────────────────────────────────────────────────────┐
│              Infrastructure (HTTP & Cache)                 │
│  ┌─────────────────────┐  ┌─────────────────────┐          │
│  │  TestIOClient       │  │  InMemoryCache      │          │
│  │  (httpx wrapper)    │  │  (TTL-based)        │          │
│  │  - Connection pool  │  │  - 1h/5m/1m TTL     │          │
│  │  - Concurrency      │  │  - Auto-expiration  │          │
│  │  - Token sanitize   │  │                     │          │
│  └──────────┬──────────┘  └─────────────────────┘          │
│             │                                              │
│             ▼                                              │
└────────────────────────────────────────────────────────────┘
             │
             ▼
┌──────────────────────────────────────────────────────────────┐
│                  TestIO Customer API                         │
│              https://api.test.io/customer/v2                 │
└──────────────────────────────────────────────────────────────┘

Key Architectural Decisions:

• Service Layer Pattern (ADR-006): Business logic is framework-agnostic and can be reused across REST APIs, CLIs, or webhooks
• FastMCP Context Injection (ADR-007): Shared resources (client, cache) initialized via lifespan handler, injected into tools via context
• Strict Type Safety: mypy --strict enforced, Pydantic v2 for validation and schema generation
• Security (SEC-002): API tokens never appear in logs or error messages, event hooks sanitize httpx logging
• Concurrency Control (ADR-002): Global semaphore limits concurrent API requests to prevent rate limiting

Technologies:

• FastMCP - Pythonic MCP framework with decorator-based API
• Pydantic v2 - Runtime validation with JSON Schema generation
• httpx - Modern async HTTP client with connection pooling
• mypy - Static type checker in strict mode

For deep dive into architecture, see docs/architecture/ARCHITECTURE.md.

---

Documentation

Comprehensive documentation is available in the docs/ directory:

Architecture & Design

• ARCHITECTURE.md - System architecture, data flow, testing strategy
• SERVICE_LAYER_SUMMARY.md - Service pattern details
• SECURITY.md - Security considerations and token sanitization
• PERFORMANCE.md - Performance targets and optimization strategies
• adrs/ - Architecture Decision Records (7 ADRs covering key technical decisions)

Development

• CLAUDE.md - Comprehensive development guide for contributors
  • Development commands (testing, code quality, running server)
  • Architecture patterns (service layer, dependency injection)
  • Adding new tools (step-by-step guide)
  • Common pitfalls and best practices

Project Planning

• stories/ - User story specifications with acceptance criteria (Stories 1-13)
• epics/ - Epic specifications
• project-brief.md - Original project vision and use cases

Contributing Guide (Coming in v0.4.0)

The contributor guide for adding new tools will be enhanced after Stories 012-013 are completed (~12.5 hours). These stories introduce:

• BaseService class for reducing boilerplate (~40% code reduction)
• get_service() helper for simplified tool creation
• FastMCP ToolError exception pattern

Until then, refer to CLAUDE.md for current development patterns.

---

Project Status

Current Version: v0.3.0 (Alpha)

MVP Complete:

• ✅ 7 MCP tools with comprehensive documentation
• ✅ Service layer architecture (framework-agnostic business logic)
• ✅ Strict type safety (mypy --strict, Pydantic v2)
• ✅ Security (token sanitization, secret scanning)
• ✅ Testing (~130 tests: unit, service, integration)
• ✅ Pre-commit hooks (ruff, mypy, detect-secrets)

Pending (v0.4.0 - ~12.5 hours):

• ⏳ Story 012: Extensibility Infrastructure
  • BaseService class for DI and caching patterns
  • get_service() context extraction helper
  • FastMCP ToolError adoption
  • Auto-discovery of tools via pkgutil
  • Impact: ~220-250 lines removed (~25% reduction in boilerplate)
• ⏳ Story 013: Usability Enhancements
  • Natural language date parsing ("last 30 days", "this quarter")
  • Filter validation with Pydantic enums
  • get_valid_bug_filters discovery tool
  • Impact: Improved UX for date-based queries

Breaking Changes in v0.4.0:

⚠️ BREAKING: list_tests default behavior changes:

• Before (v0.3.0): Default returns only running tests
• After (v0.4.0): Default returns ALL tests (no filtering)

See CHANGELOG.md for full version history.

Roadmap:

• v0.4.0: Extensibility & usability improvements (Stories 012-013)
• v0.5.0: Write operations (create tests, accept/reject bugs) - requires authentication strategy
• v0.6.0: Multi-tenant support (per-request tokens) - requires HTTP transport refactoring

---

Development

Running Tests

# Run all tests (unit + integration with API token)
uv run pytest

# Run specific test types
uv run pytest -m unit              # Fast unit tests (no API needed)
uv run pytest -m integration       # Integration tests (requires API credentials)

# Run with coverage
uv run pytest --cov=src/testio_mcp --cov-report=html

Integration Tests:

Some integration tests require TESTIO_TEST_ID environment variable for positive test cases:

export TESTIO_TEST_ID=109363  # Use a valid test ID from your account
uv run pytest -m integration

Code Quality

# Run linter (auto-fix)
uv run ruff check --fix

# Run formatter
uv run ruff format

# Run type checker
uv run mypy src/testio_mcp

# Run all pre-commit hooks manually
pre-commit run --all-files

Testing Tools via MCP Inspector

# List available tools
npx @modelcontextprotocol/inspector --cli uv run python -m testio_mcp --method tools/list

# Test health_check (no parameters)
npx @modelcontextprotocol/inspector --cli uv run python -m testio_mcp --method tools/call --tool-name health_check

# Test get_test_status (with parameter)
npx @modelcontextprotocol/inspector --cli uv run python -m testio_mcp --method tools/call --tool-name get_test_status --tool-arg 'test_id=109363'

# Interactive mode (opens web UI at localhost:6274)
npx @modelcontextprotocol/inspector uv run python -m testio_mcp

Important: Integer parameters must be passed as integers, not quoted strings.

For detailed development workflows, see CLAUDE.md.

---

License & Disclaimer

License: Proprietary (see LICENSE)

This software is provided by TestIO for use "AS IS" without warranty of any kind. By using this software, you agree to the following terms:

• ✅ Free to Use: Personal and commercial use permitted
• ❌ No Warranty: No guarantees of fitness, reliability, or accuracy
• ❌ No Liability: TestIO is not liable for any damages or issues arising from use
• ❌ No Support: TestIO has no obligation to provide support, updates, or bug fixes
• ❌ No Modification/Redistribution: You may not modify, adapt, or redistribute this software

For complete license terms, see the LICENSE file.

---

Configuration

All configuration via environment variables in .env file:

# Required
TESTIO_CUSTOMER_API_TOKEN=your-token-here
TESTIO_CUSTOMER_API_BASE_URL=https://api.test.io/customer/v2

# Optional (defaults shown)
MAX_CONCURRENT_API_REQUESTS=10
CONNECTION_POOL_SIZE=100
CONNECTION_POOL_MAX_KEEPALIVE=20
HTTP_TIMEOUT_SECONDS=30
LOG_LEVEL=INFO
LOG_FORMAT=text  # or "json"

# Tool Control (optional)
ENABLED_TOOLS=health_check,list_products  # Allowlist: only these tools available
DISABLED_TOOLS=generate_status_report     # Denylist: all except these tools
# Note: Cannot use both ENABLED_TOOLS and DISABLED_TOOLS simultaneously

Tool Enable/Disable

Control which MCP tools are available by using environment variables:

Allowlist mode (ENABLED_TOOLS):

# Only enable specific tools (all others disabled)
ENABLED_TOOLS=health_check,list_products,get_test_status

Denylist mode (DISABLED_TOOLS):

# Disable specific tools (all others enabled)
DISABLED_TOOLS=generate_status_report,get_test_activity_by_timeframe

Available tool names:

• health_check - Verify API authentication
• get_cache_stats - Cache performance metrics
• clear_cache - Clear cached data
• list_products - List all products
• list_tests - List tests for a product
• get_test_status - Get test status details
• get_test_bugs - Get bug details with filtering
• generate_status_report - Generate executive summary
• get_test_activity_by_timeframe - Query test activity by date range

Format options:

• Comma-separated: ENABLED_TOOLS=tool1,tool2,tool3
• JSON array: ENABLED_TOOLS='["tool1", "tool2", "tool3"]'

Use cases:

• CSM workflow: ENABLED_TOOLS=health_check,get_test_status,list_tests
• QA workflow: ENABLED_TOOLS=get_test_bugs,generate_status_report,list_tests
• Minimal setup: DISABLED_TOOLS=generate_status_report,get_test_activity_by_timeframe

See .env.example for complete configuration options.

---