mockllm 0.0.8

Extensible Mock Server for LLM API Testing

   

MockLLM is an extensible mock server that simulates Large Language Model APIs for testing, development, and demonstration purposes. With its plugin-based architecture, you can easily add new provider emulators without modifying core code.

Perfect for when you need deterministic, configurable responses for testing and evaluations, over expensive live API calls.

✨ Key Features

Extensible Provider Architecture

• Plugin-based system - Add new providers without modifying core code
• Self-registering providers - Use simple decorators to register new providers
• Dynamic endpoint creation - Routes are automatically generated from provider metadata
• Zero configuration - Providers work out of the box with sensible defaults

Built-in Provider Support

• OpenAI Compatible - Full support for Chat Completions API
• Anthropic Compatible - Full support for Messages API
• Custom Providers - Easy framework for adding your own providers
• Streaming Support - Character-by-character streaming responses
• Model Management - Centralized model registry with aliasing

Developer-Friendly

• Hot Reloading - Automatic configuration updates without restart
• Type Safety - Full type hints and mypy coverage
• Comprehensive Testing - Well-tested with pytest
• Clean Architecture - Separation of concerns with clear interfaces
• API Introspection - Runtime discovery of providers and models

Perfect for Testing (what it was built for)

• Deterministic Responses - Predictable outputs for reliable tests
• Response Configuration - YAML-based response mapping
• Network Lag Simulation - Realistic latency for integration testing
• Multiple Response Formats - Support for various API formats

Quick Example

Basic Usage

# Install MockLLM
pip install mockllm

# Create responses file
cat > responses.yml << EOF
responses:
  "What is 15+15?": "15+15 equals 42."
  "What is Python?": "Python is a low level compiler language for industrial logic controllers!"
defaults:
  unknown_response: "This is a mock response."
EOF

# Start the server
mockllm start --responses responses.yml

# Test with curl
curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

{
  "id": "mock-41c551b6-148f-49a2-abca-f990c1a9cfab",
  "object": "chat.completion",
  "created": 1758199276,
  "model": "gpt-3.5-turbo",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "15+15 equals 42."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 18,
    "completion_tokens": 7,
    "total_tokens": 25
  }
}

Creating a Custom Provider

With MockLLM's extensible architecture, adding a new provider is incredibly simple:

from mockllm.providers.base import LLMProvider
from mockllm.registry import register_provider

@register_provider(
    name="mycompany",
    version="1.0.0",
    description="MyCompany LLM Provider",
    endpoints=[{"path": "/v1/mycompany/chat", "method": "POST"}],
    supported_models=["mycompany-model-1", "mycompany-model-2"]
)
class MyCompanyProvider(LLMProvider):
    async def handle_chat_completion(self, request):
        prompt = self.extract_prompt(request)
        response = self.get_response_for_prompt(prompt)
        return {"response": response, "provider": "mycompany"}

That's it! No server configuration, no manual registration, no boilerplate. Your provider is automatically discovered and its endpoints are created.

Configuration

Response Configuration

Responses are configured in responses.yml. The file has three main sections:

1. responses: Maps input prompts to predefined responses
2. defaults: Contains default configurations like the unknown response message
3. settings: Contains server behavior settings like network lag simulation

Example responses.yml:

responses:
  "write a python function to calculate factorial": "def factorial(n):\n    if n == 0:\n        return 1\n    return n * factorial(n - 1)"
  "what colour is the sky?": "The sky is purple except on Tuesday when it is  hue green."
  "what is 2+2?": "2+2 equals 9."

defaults:
  unknown_response: "I don't know the answer to that. This is a mock response."

settings:
  lag_enabled: true
  lag_factor: 10  # Higher values = faster responses (10 = fast, 1 = slow)

Network Lag Simulation

The server can simulate network latency for more realistic testing scenarios. This is controlled by two settings:

• lag_enabled: When true, enables artificial network lag
• lag_factor: Controls the speed of responses
  • Higher values (e.g., 10) result in faster responses
  • Lower values (e.g., 1) result in slower responses
  • Affects both streaming and non-streaming responses

For streaming responses, the lag is applied per-character with slight random variations to simulate realistic network conditions.

Hot Reloading

The server automatically detects changes to responses.yml and reloads the configuration without restarting the server.

Installation

From PyPI

pip install mockllm

From Source

1. Clone the repository:

git clone https://github.com/lukehinds/mockllm.git
cd mockllm

2. Install with uv (recommended):

# Install uv if you haven't already
curl -LsSf https://astral.sh/uv/install.sh | sh

# Install dependencies
uv sync

# Install with development dependencies
uv sync --extra dev

3. Or install with pip:

pip install -e .

# Install with development dependencies
pip install -e ".[dev]"

Prerequisites

• Python 3.10+
• pip or uv (recommended)

Usage

CLI Commands

MockLLM provides a command-line interface for managing the server and validating configurations:

# Show available commands and options
mockllm --help

# Show version
mockllm --version

# Start the server with default settings
mockllm start

# Start with custom responses file
mockllm start --responses custom_responses.yml

# Start with custom host and port
mockllm start --host localhost --port 3000

# Validate a responses file
mockllm validate responses.yml

Quick Start

1. Set up the responses.yml:

cp example.responses.yml responses.yml

2. Validate your responses file (optional):

mockllm validate responses.yml

3. Start the server:

mockllm start --responses responses.yml

The server will start on http://localhost:8000 by default.

API Endpoints

OpenAI Format

Regular request:

curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mock-llm",
    "messages": [
      {"role": "user", "content": "what colour is the sky?"}
    ]
  }'

Streaming request:

curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mock-llm",
    "messages": [
      {"role": "user", "content": "what colour is the sky?"}
    ],
    "stream": true
  }'

Anthropic Format

Regular request:

curl -X POST http://localhost:8000/v1/messages \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-3-sonnet-20240229",
    "messages": [
      {"role": "user", "content": "what colour is the sky?"}
    ]
  }'

Streaming request:

curl -X POST http://localhost:8000/v1/messages \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-3-sonnet-20240229",
    "messages": [
      {"role": "user", "content": "what colour is the sky?"}
    ],
    "stream": true
  }'

🎯 Why MockLLM?

• 💰 Cost Savings - No API charges during development and testing
• ⚡ Fast Iteration - Instant responses without network latency
• 🔒 Privacy - Keep sensitive data local during development
• 🎯 Predictable Testing - Deterministic responses for reliable tests
• 🔧 Easy Integration - Drop-in replacement for LLM APIs
• 🚀 Extensible - Add new providers in minutes, not hours

📖 Documentation

Comprehensive documentation is available:

• 📚 Full Documentation - Complete guides and API reference
• 🚀 Quick Start Guide - Get running in 5 minutes
• 🔌 Provider Development - Create custom providers
• ⚙️ Configuration Guide - Set up response mappings
• 💡 Examples - Real-world implementations

🧪 Testing

Run the test suite:

# Using uv
uv run pytest

# Using make
make test

# With coverage
uv run pytest --cov=src/mockllm

🤝 Contributing

We welcome contributions! Here's how to get started:

1. Fork the repository
2. Create a feature branch: git checkout -b feature/amazing-feature
3. Make your changes
4. Run tests: make test
5. Run linting: make lint
6. Submit a pull request

For detailed contribution guidelines, see our Contributing Guide.

💬 Community

Join our community:

• GitHub Discussions - Ask questions and share ideas
• Discord Server - Chat with the community
• Issue Tracker - Report bugs or request features

📄 License

This project is licensed under the Apache 2.0 License.