jleechanorg-testing-utils

Generic server/MCP/browser testing utilities for Python projects.

Features

• HTTP client with request/response capture, credential redaction, and retry logic
• MCP client for Model Context Protocol (JSON-RPC 2.0) servers
• Evidence bundles – structured /tmp/<project>/<branch>/iteration_NNN/ directories with git provenance
• Local server management – subprocess server lifecycle with port reservation
• Generic test base – abstract BaseTestRunner for server integration tests
• Browser test base – Playwright-based BrowserTestBase for UI tests
• HTTP test base – HttpTestBase using requests/httpie for web server tests
• Server-side logging – LLMCallLogger and HttpRequestLogger that servers can import to capture raw request/response payloads

Installation

pip install jleechanorg-testing-utils

# For browser testing:
pip install "jleechanorg-testing-utils[browser]"

Usage

MCP Server Testing

from testing_utils import MCPClient, BaseTestRunner, TestResult

class SmokeTest(BaseTestRunner):
    TEST_NAME = "mcp_smoke"
    PROJECT_NAME = "myproject"
    DEFAULT_BASE_URL = "http://localhost:8081"

    def run_scenarios(self) -> list[TestResult]:
        tools = self.client.tools_list()
        return [TestResult("tools_list", passed=bool(tools), detail=f"{len(tools)} tools")]

if __name__ == "__main__":
    SmokeTest().run()

HTTP Web Server Testing

from testing_utils.http_test import HttpTestBase

class ApiTest(HttpTestBase):
    TEST_NAME = "api_smoke"
    BASE_URL = "http://localhost:9000"
    PROJECT_NAME = "myproject"

    def run_scenarios(self):
        r = self.get("/health")
        r.assert_ok()
        r.assert_json_key("status", "healthy")

        # httpie-style subprocess
        rc, out, err = self.httpie("GET", "/api/users")
        return [{"name": "health_ok", "passed": r.ok}]

if __name__ == "__main__":
    test = ApiTest()
    results = test.run()

Browser Testing (requires [browser] extra)

from testing_utils.browser import BrowserTestBase

class HomepageTest(BrowserTestBase):
    BASE_URL = "http://localhost:3000"
    TEST_NAME = "homepage_smoke"
    PROJECT_NAME = "myproject"
    HEADLESS = True

    def run_scenarios(self):
        self.page.goto(self.BASE_URL)
        title = self.page.title()
        self.take_screenshot("homepage")
        return [{"name": "page_loads", "passed": bool(title), "detail": title}]

if __name__ == "__main__":
    HomepageTest().run()

Server-Side LLM Logging

Import in your server to capture raw LLM request/response payloads for evidence bundles:

from testing_utils.logging_capture import LLMCallLogger

# Use default env var names:
_llm_logger = LLMCallLogger(
    capture_path_env="RAW_LLM_CAPTURE_PATH",
    enabled_env="CAPTURE_RAW_LLM",
    max_chars_env="CAPTURE_RAW_LLM_MAX_CHARS",
)

def call_llm(provider, model, messages, system_instruction=None):
    _llm_logger.log_request(
        provider=provider,
        model=model,
        request_payload={"messages": messages},
        system_instruction=system_instruction,
    )
    response_text = _actual_llm_call(provider, model, messages)
    _llm_logger.log_response(
        provider=provider,
        model=model,
        response_text=response_text,
    )
    return response_text

Set RAW_LLM_CAPTURE_PATH=/tmp/myproject/llm_captures.jsonl before running tests to enable capture.

Evidence Bundles

from testing_utils.evidence import (
    get_evidence_dir,
    create_evidence_bundle,
    write_with_checksum,
    capture_provenance,
)

# Get branch-scoped evidence directory
evidence_dir = get_evidence_dir("my_test", project_name="myproject")
# → /tmp/myproject/<branch>/my_test/

# Create a structured bundle after running tests
create_evidence_bundle(
    evidence_dir,
    results={"passed": 5, "failed": 0},
    test_name="my_test",
    project_name="myproject",
    http_captures=client.get_captures_as_dict(),
)
# → /tmp/myproject/<branch>/my_test/iteration_001/
#   ├── README.md
#   ├── metadata.json
#   ├── results.json
#   └── request_responses.jsonl

Environment Variables

Design Principles

• No hardcoded project names – everything is parameterized
• No hardcoded URLs or credentials – all configurable via constructor or env vars
• Thread-safe – logging capture uses file locks
• Credential redaction – API keys, JWTs, and auth tokens are automatically redacted in captures
• Evidence-first – every test run creates a structured evidence bundle with git provenance