lmux

Core types, protocols, and utilities for the lmux ecosystem.

You don't need to install this directly; provider packages (e.g., lmux-openai) include it as a dependency. Install it only if you're building a custom provider.

Types

Messages

• SystemMessage: system/instruction message
• DeveloperMessage: developer message (for o-series models)
• UserMessage: user message, supports text and multimodal content (TextContent, ImageContent, CachePointContent)
• AssistantMessage: assistant message with optional tool calls and provider continuation state
• ToolMessage: tool result

Responses

• ChatResponse: chat completion result with content, usage, cost, model, provider, finish_reason, optional provider_metadata, and optional provider continuation state
• ChatChunk: streaming chunk with delta, tool_call_deltas, usage, cost, plus the serving model/provider stamped on the terminal chunk and optional metadata and continuation state
• EmbeddingResponse: embedding result with embeddings, usage, cost
• ResponseResponse: Responses API result with output_text, usage, cost
• ResponseInputMessage: Responses API message input, supporting text and multimodal content (TextContent, ImageContent, CachePointContent)

Provider continuations

Some provider protocols return opaque, signed state that must be sent back unchanged to continue a tool-use turn. Providers that support this attach a ProviderContinuation to the response and terminal stream chunk. Use response.to_assistant_message() to preserve it:

response = provider.chat(model, messages, tools=tools)
messages.append(response.to_assistant_message())
messages.append(ToolMessage(content=tool_result, tool_call_id=tool_call_id))

Each provider package owns its continuation namespace and payload. When an assistant message returns to the matching provider, that provider may treat the continuation as authoritative instead of rebuilding the turn from normalized content and tool calls. Other providers ignore it and use the normalized fields, so the message remains portable.

Treat continuation data as opaque: preserve it unchanged or set it to None. It can include full reasoning text and signatures, may be large or sensitive, and is included by model_dump() and model_dump_json().

Cost

• Usage: token counts (input_tokens, output_tokens, cache_read_tokens, cache_creation_tokens)
• Cost: cost breakdown (input_cost, output_cost, total_cost, plus cache costs)
• ModelPricing / PricingTier: tiered pricing configuration
• PricingSchedule: dated price overrides for models whose rate changes on a known date (e.g. an introductory rate)
• per_million_tokens(): converts per-million price to per-token price
• calculate_cost(): calculates cost from usage and pricing; pass as_of=<date> to bill at the rate in effect on that day (defaults to the latest schedule)

Tools

• Tool: function tool definition
• ToolChoice / ToolChoiceFunction: control whether and which tool the model calls
• ToolCall / ToolCallDelta: tool call in responses and streaming
• FunctionDefinition / FunctionCallResult / FunctionCallDelta

Response Format

• TextResponseFormat / JsonObjectResponseFormat / JsonSchemaResponseFormat

Protocols

from lmux import CompletionProvider, EmbeddingProvider, ResponsesProvider, PricingProvider, AsyncCloseable

• CompletionProvider[ParamsT]: chat, achat, chat_stream, achat_stream
• EmbeddingProvider[ParamsT]: embed, aembed
• ResponsesProvider[ParamsT]: create_response, acreate_response
• PricingProvider: register_pricing
• AuthProvider[AuthT]: get_credentials, aget_credentials
• AsyncCloseable: aclose

All are @runtime_checkable, so you can use isinstance() to check support.

Registry

Route "prefix/model" strings to provider instances:

from lmux import Registry

registry = Registry()
registry.register("openai", openai_provider)
registry.register("anthropic", anthropic_provider)

response = registry.chat("openai/gpt-4o", messages)
response = registry.chat("anthropic/claude-sonnet-4-20250514", messages)

# Close all providers that implement AsyncCloseable
await registry.aclose()

registry.registered_prefixes returns a frozenset of the registered prefixes, useful for a composite provider that validates its delegation targets.

Exceptions

All exceptions inherit from LmuxError and carry optional provider and status_code fields:

• AuthenticationError
• RateLimitError (with retry_after)
• InvalidRequestError
• NotFoundError
• ProviderError
• TimeoutError
• UnsupportedFeatureError

MockProvider

Built-in mock for testing. Implements all protocols with configurable responses and call tracking.

from lmux import MockProvider, ChatResponse

mock = MockProvider(chat_responses=[ChatResponse(...)])
response = mock.chat("any-model", messages)
assert len(mock.calls) == 1