LLM API Adapter SDK for Python

Overview

llm-api-adapter is a minimal, typed adapter for nine organizations: OpenAI, Anthropic, Google, Mistral, xAI, Qwen, Kimi, DeepSeek, and Z.ai. It provides one provider-neutral contract for messages, tools, structured output, multimodal input, errors, usage, cost, and streaming — with one runtime dependency and no provider SDKs or orchestration framework. Switching organizations means changing two arguments.

Note: Mistral, xAI, Qwen, Kimi, DeepSeek, and Z.ai are installed separately with their optional extras.

Supports Python 3.10–3.14.

Contents

• Overview
• Why this library?
• Features
• Installation
• Getting Started
• Model-specific request compatibility
• Streaming
• Async API
• Optional HTTPX Sync Transport
• Handling Errors
• Configuration and Management
• Example Use Case
• Timeout Support
• Reasoning Support
• Tool / Function Calling
• Structured Output
• Vision Input
• Document Input
• Token Usage and Pricing
• Logging
• Adoption
• Related project
• Development & Testing
• License

Why this library?

	llm-api-adapter	LiteLLM	LangChain	Provider SDK
Runtime dependencies	1 (requests)	broader gateway/SDK	broader framework	provider-specific
Provider-neutral messages and response	✓	✓	✓	✗
Unified tool calls	✓	✓	✓	provider-specific
Unified structured output	✓	✓	✓	provider-specific
Unified image/PDF input	✓	partial	✓	provider-specific
Unified reasoning control	✓*	✓	partial	✗
Built-in per-response cost	✓	✓	via callbacks	✗
Unified error hierarchy	✓	OpenAI-compatible	framework-specific	✗
Sync streaming	✓ text-first	✓	✓	✓
Async API	✓ optional	✓	✓	✓
Number of organizations	9	100+	50+	1

• reasoning_level is one application-level parameter, but the available levels, native mapping, and emitted reasoning content remain model/provider-dependent.

This table is a positioning snapshot. Provider capabilities change independently, so the adapter promises a stable application-level contract rather than identical provider internals.

Small footprint. The core has one runtime dependency: requests. It does not require OpenAI, Anthropic, or Google SDKs, an agent framework, a gateway, or a proxy. Pydantic is optional and is only needed for response_model validation.

One provider-neutral contract. The same typed messages, ToolSpec, ChatResponse, ImagePart, and DocumentPart are converted to and from each provider's native wire format. Application code does not need provider-specific message, tool, or response parsing.

Model-aware reasoning control. One reasoning_level parameter is accepted across supported providers without provider-specific kwargs. Verified model capabilities resolve the canonical level to a native effort or budget setting; exact availability and token semantics remain model-dependent.

Cost accounting, built-in. Responses carry normalized usage and cost fields in the configured currency. When a provider reports an exact total that cannot be split safely, cost_total is set while cost_input and cost_output remain unset. Organization packages report separately metered, non-token charges as CostLineItem values in cost_breakdown.

Predictable errors. One explicit exception hierarchy and provider-error mapping across supported organizations. LLMAPIRateLimitError means the same application-level condition whether you called OpenAI, Anthropic, Google, Mistral, or xAI.

Use this when

• You call LLMs directly — no chains, no agents, no orchestration
• You want a provider-neutral contract without adopting a framework or gateway
• You need the same messages, tools, structured output, multimodal input, errors, and response fields across providers
• You need unified reasoning control or per-request cost tracking with minimal dependencies

Use something else when

• LiteLLM — you need providers beyond OpenAI, Anthropic/Claude, and Google, or a gateway/router layer with retries, fallbacks, load balancing, and observability
• AISuite — you prefer an OpenAI-compatible client across more providers, or built-in agents, MCP, and automatic tool-execution abstractions
• LangChain — you need chains, memory, RAG, agents, or a larger orchestration framework
• Provider SDK directly — you use one provider and need an API or feature outside this SDK's provider-neutral contract

Features

• Synchronous Streaming: Iterate normalized text with stream_chat() across supported organizations. Optionally coalesce it into bounded chunks and observe per-chunk metadata without changing the yielded str contract.
• Optional HTTPX Sync Transport: Keep requests as the default synchronous transport or opt into HTTPX with transport="httpx". See the HTTPX sync pilot guide.
• Asynchronous API: Use achat() and astream_chat() with the optional [async] installation for non-blocking HTTPX requests and awaitable callbacks. See the Async API guide.
• Reasoning Observability: Opt in to provider-emitted reasoning summaries through capture_reasoning=True, ReasoningEvent, and the on_reasoning callback without mixing reasoning into visible text.
• Provider-Neutral Messages and Responses: Use the same typed messages, ChatResponse, usage, pricing, parsed output, and tool-call fields regardless of the provider.
• Vision Input: Send images alongside text via ImagePart as a URL, raw bytes, or data URI when the selected organization supports that form.
• PDF Documents: Send PDF URLs or bytes via DocumentPart; provider-specific file/document payloads are generated automatically.
• Tool / Function Calling: Provider-agnostic tool definitions and normalized tool calls in ChatResponse.tool_calls.
• Portable Structured Output: Pass a documented portable JSON Schema to chat() and get a parsed object in ChatResponse.parsed_json across all supported organizations.
• Pydantic Integration: Pass a portable Pydantic model as response_model and get a typed instance back in ChatResponse.parsed_model — no manual schema writing required.
• Request Timeouts: Per-request timeout control via timeout_s; raises LLMAPITimeoutError on expiry.
• Flexible Configuration: temperature, max_tokens, top_p, and other parameters are translated to a verified provider payload; known unsupported fields are omitted predictably.
• Pricing Registry: Model prices stored in a bundled JSON registry with per-model input/output rates; overridable per instance.

Installation

To install the SDK, you can use pip:

pip install llm-api-adapter

To use Mistral, install its optional organization package:

pip install "llm-api-adapter[mistral]"

To use xAI, install its optional organization package:

pip install "llm-api-adapter[xai]"

To use Qwen Model Studio, install its optional organization package:

pip install "llm-api-adapter[qwen]"

To use Kimi / Moonshot, install its optional organization package:

pip install "llm-api-adapter[kimi]"

To use DeepSeek, install its optional organization package:

pip install "llm-api-adapter[deepseek]"

To use Z.ai / GLM, install its optional organization package:

pip install "llm-api-adapter[zai]"

The Mistral package README, xAI package README, Qwen package README, Kimi package README, DeepSeek package README, and Z.ai package README list their supported models and organization-specific behaviour. Direct installation of llm-api-adapter-mistral, llm-api-adapter-xai, llm-api-adapter-qwen, llm-api-adapter-kimi, llm-api-adapter-deepseek, or llm-api-adapter-zai remains supported.

Core baseline and organization packages. An organization enters Core when its supported models and adapter implement the complete provider-neutral baseline: typed multi-turn messages and roles; sync/async chat and streaming; image and PDF input; tools; portable structured output; common sampling, output-limit, and timeout parameters; and normalized ChatResponse, errors, usage, and token pricing. The shared conformance suite verifies this contract.

Optional organization packages own their adapters, registries, and provider-specific preprocessing. Mistral PDF input is one such extension: the package calls Mistral OCR before chat because OCR is not part of the selected chat model, then passes the resulting Markdown to it. That page-based OCR charge is a CostLineItem, not a token cost. See the Mistral package README for its exact behaviour and pricing.

For non-blocking requests, install the optional [async] extra and follow the Async API guide.

To opt into HTTPX for synchronous chat() and stream_chat() calls, install the optional [httpx] extra and pass transport="httpx". The default remains requests; see the HTTPX sync pilot guide.

Note: You need an API key from each LLM provider you use, including Mistral, xAI, Qwen, Kimi, DeepSeek, or Z.ai when their optional packages are installed. Refer to the provider's documentation for API-key instructions.

Getting Started

Importing and Setting Up the Adapter

To start using the adapter, you need to import the necessary components:

from llm_api_adapter.models.messages.chat_message import (
    AIMessage, Prompt, UserMessage
)
from llm_api_adapter.universal_adapter import UniversalLLMAPIAdapter

Sending a Simple Request

The SDK supports three types of messages for interacting with the LLM:

• Prompt: Use Prompt to set the context or initial prompt for the model.
• UserMessage: Use UserMessage to send messages from the user during a conversation.
• AIMessage: Use AIMessage to simulate responses from the assistant during a conversation.

Here is an example of how to send a simple request to the adapter:

messages = [
    UserMessage("Hi! Can you explain how artificial intelligence works?")
]

adapter = UniversalLLMAPIAdapter(
    organization="openai",
    model="gpt-5",
    api_key=openai_api_key
)

response = adapter.chat(
    messages=messages,
    max_tokens=max_tokens,
    temperature=temperature,
    top_p=top_p
)
print(response.content)

Parameters

• max_tokens: The maximum number of tokens to generate in the response. This limits the length of the output.

• temperature: Controls the randomness of the response. Higher values (e.g., 0.8) make the output more random, while lower values (e.g., 0.2) make it more focused and deterministic. Default value: 1.0 (range: 0 to 2).

• top_p: Limits the response to a certain cumulative probability. This is used to create more focused and coherent responses by considering only the highest probability options. Default value: 1.0 (range: 0 to 1).

Model-specific request compatibility

The public chat(), achat(), stream_chat(), and astream_chat() methods keep the same arguments across providers. For a verified model, the adapter applies its registered payload compatibility rules before making the request; sync and async calls use the same rules.

If a rule omits a parameter, its documented default is removed silently. A different supplied value is also removed, but emits one UserWarning and one WARNING log record per ignored parameter. Treat that warning as a prompt to remove the setting or select a model that supports it.

Rules match the exact model names in the bundled registry, with two narrow snapshot exceptions for direct provider APIs: Anthropic claude-...-YYYYMMDD and OpenAI gpt-...-YYYY-MM-DD IDs inherit a registered unsuffixed base model's metadata when the date is valid. The requested snapshot ID is still sent to the provider. The adapter does not infer behavior from a name prefix; Google aliases and preview IDs, fine-tuned IDs, and any snapshot whose base model is unregistered receive no compatibility transformation. An unknown OpenAI model uses Chat Completions rather than the Responses API. Use a listed model or request a verified registry addition when a provider adds a new alias or model.

For gemini-3.7-flash, gemini-3.6-flash, and gemini-3.5-flash-lite, Google does not support sampling controls. The adapter omits temperature and top_p from these model requests; non-default values produce the compatibility warning described above.

gpt-6-astra uses the OpenAI Responses API and does not support temperature or top_p; the adapter omits both according to the same warning policy. Astra also cannot disable reasoning: reasoning_level="none" resolves to its lowest supported effort, low, with a UserWarning.

gpt-6-sol and gpt-6-luna use the Responses API and support reasoning_level="none". With any higher reasoning effort, OpenAI does not accept temperature or top_p; the adapter omits them using the same warning policy.

Alternative Message Format

In addition to the built-in message classes, the SDK also supports the standard OpenAI-style message format for quick adoption and compatibility:

messages = [
    {"role": "system", "content": "You are a friendly assistant who answers only yes or no."},
    {"role": "user", "content": "Do you know how AI learns?"},
    {"role": "assistant", "content": "Yes."},
    {"role": "user", "content": "Can you explain it in one sentence?"}
]

response = adapter.chat(messages=messages, max_tokens=50)
print(response.content)

Note
The adapter automatically normalizes message input — you can mix custom message classes and OpenAI-style dicts in one list.

Streaming

stream_chat() is the synchronous streaming counterpart to chat(). It always yields normalized visible text as str, independently of the provider's native streaming event format.

By default, buffer_chars=None preserves provider text-delta behavior. Set a positive buffer_chars value when a consumer prefers bounded, coalesced text; on_chunk then receives a StreamChunk with the same text and observability metadata.

from llm_api_adapter.universal_adapter import UniversalLLMAPIAdapter

adapter = UniversalLLMAPIAdapter(
    organization="openai",
    model="gpt-5.6-sol",
    api_key="...",
)

chunk_metadata = []
for text in adapter.stream_chat(
    messages=[{"role": "user", "content": "Explain SSE in one sentence."}],
    buffer_chars=80,
    on_chunk=chunk_metadata.append,
    on_tool_call=lambda call: print(call.name, call.arguments),
    on_done=lambda response: print(response.usage),
):
    print(text, end="", flush=True)  # `text` is still a str.

for chunk in chunk_metadata:
    print(chunk.index, chunk.elapsed_s, chunk.usage, chunk.output_tokens_delta)

• buffer_chars accepts None (the default) or a positive integer. Buffered chunks never exceed the configured size; any remaining text is emitted during normal completion.
• on_chunk(chunk) receives a StreamChunk with text, monotonic index, local elapsed_s / delta_s, and optional usage / output_tokens_delta fields.
• on_delta(text) is called for every yielded visible text chunk. The order is always on_chunk → on_delta → yield.
• on_tool_call(tool_call) receives only completed, normalized ToolCall objects after the provider stream finishes.
• on_done(response) receives the finalized ChatResponse, including usage, pricing, parsed structured output, and any tool calls, after the final buffer flush.
• Token metadata is optional and comes only from provider usage payloads. When a provider reports cumulative output usage, output_tokens_delta is the local increment; no token estimation is performed.

Buffering is pull-based and has no background worker or time-based flush. If the stream fails or a caller closes the iterator early, pending text is not emitted as a successful final chunk and on_done is not called.

stream_chat() uses the synchronous requests transport by default. To opt into HTTPX for both chat() and stream_chat(), see the HTTPX sync pilot guide. For the matching astream_chat() contract and its httpx.AsyncClient transport, see the Async API guide.

Provider event references: OpenAI Responses streaming, Anthropic streaming, and Google streamGenerateContent.

Async API

The optional asynchronous client, installation command, achat() and astream_chat() examples, callback ordering, cancellation behavior, and reasoning observability are documented in the Async API guide.

Optional HTTPX Sync Transport

The default synchronous transport is requests and requires no additional dependency. HTTPX is available as an opt-in pilot for synchronous chat() and stream_chat() calls:

pip install "llm-api-adapter[httpx]"

adapter = UniversalLLMAPIAdapter(
    organization="openai",
    model="gpt-5",
    api_key="...",
    transport="httpx",
)
response = adapter.chat([UserMessage("Say hello")])

See the HTTPX sync pilot guide for compatibility, limitations, and E2E verification details.

Handling Errors

Common Errors

The SDK provides a set of standardized errors for easier debugging and integration:

API Errors

• LLMAPIError: Base class for all API-related errors. This error is also used for any unexpected LLM API errors.

• LLMAPIAuthorizationError: Raised when authentication or authorization fails.

• LLMAPIRateLimitError: Raised when rate limits are exceeded.

• LLMAPITokenLimitError: Raised when token limits are exceeded.

• LLMAPIClientError: Raised when the client makes an invalid request.

• LLMAPIServerError: Raised when the server encounters an error.

• LLMAPITimeoutError: Raised when a request times out.

• LLMAPIUsageLimitError: Raised when usage limits are exceeded.

• InvalidToolSchemaError: Raised when a provided tool schema is invalid.

• InvalidToolArgumentsError: Raised when tool arguments cannot be parsed or validated.

• ToolChoiceError: Raised when tool_choice is invalid or references an unknown tool.

• JSONSchemaError: Raised for incompatible structured-output arguments, a non-portable or non-recursive schema, an invalid Pydantic response model, invalid JSON, or failed Pydantic response validation.

Config Errors

• LLMConfigError: Raised when the request configuration is invalid or incompatible.

• LLMReasoningLevelError: Raised only for Anthropic models when max_tokens is less than reasoning_level.

Provider Error Mapping

Exception	OpenAI	Anthropic	Google	Kimi
LLMAPIAuthorizationError	HTTP 401; InvalidAuthenticationError, AuthenticationError	HTTP 401; AuthenticationError, PermissionError	HTTP 401/403; PERMISSION_DENIED	HTTP 401/403; authentication or permission error type
LLMAPIRateLimitError	HTTP 429; RateLimitError	HTTP 429; RateLimitError	HTTP 429; RESOURCE_EXHAUSTED	HTTP 429; rate_limit_error, rate_limit_exceeded
LLMAPITokenLimitError	MaxTokensExceededError, TokenLimitError	—	—	context_length_exceeded, input_too_long, output-token limit types
LLMAPIClientError	HTTP 4xx; InvalidRequestError, BadRequestError	HTTP 4xx; InvalidRequestError, RequestTooLargeError, NotFoundError	HTTP 4xx; INVALID_ARGUMENT, FAILED_PRECONDITION, NOT_FOUND	Unclassified HTTP/client or SSE error
LLMAPIServerError	HTTP 5xx; InternalServerError, ServiceUnavailableError	HTTP 5xx; APIError, OverloadedError	HTTP 5xx; INTERNAL, UNAVAILABLE	HTTP 5xx; api_error, internal_error, overloaded_error
LLMAPITimeoutError	requests.Timeout, httpx.TimeoutException; TimeoutError	requests.Timeout, httpx.TimeoutException	requests.Timeout, httpx.TimeoutException; DEADLINE_EXCEEDED	HTTP 408/504; timeout, timeout_error
LLMAPIUsageLimitError	UsageLimitError, QuotaExceededError	—	—	insufficient_quota, quota_exceeded, usage_limit_exceeded

• Kimi maps its documented token-limit and quota error types to LLMAPITokenLimitError and LLMAPIUsageLimitError. Equivalent cases for Anthropic and Google fall into LLMAPIClientError (HTTP 4xx).
• LLMAPIClientError is the default fallback for all unhandled HTTP 4xx responses.
• LLMAPIServerError is the default fallback for all HTTP 5xx responses.
• InvalidToolSchemaError, InvalidToolArgumentsError, ToolChoiceError, JSONSchemaError — client-side errors (validated before the request is sent); inherit from LLMAPIClientError.
• LLMConfigError, LLMReasoningLevelError — configuration errors (parameter validation before the request); LLMReasoningLevelError is only raised for Anthropic models with budget_tokens.

Configuration and Management

Using Different Providers and Models

The SDK allows you to easily switch between LLM providers and specify the model you want to use. Currently supported providers are OpenAI, Anthropic, Google, Mistral, xAI, Qwen, Kimi, DeepSeek, and Z.ai. Mistral, xAI, Qwen, Kimi, DeepSeek, and Z.ai require their corresponding optional extras.

• OpenAI: You can use models like gpt-6-astra, gpt-6-sol, gpt-6-luna, gpt-5.6-sol, gpt-5.6-terra, gpt-5.6-luna, gpt-5.5, gpt-5.4, gpt-5.4-mini, gpt-5.4-nano, gpt-5.2, gpt-5.1, gpt-5, gpt-5-mini, gpt-5-nano, gpt-4.1, gpt-4.1-mini, gpt-4.1-nano, gpt-4o, gpt-4o-mini.
• Anthropic: Available models include claude-fable-5-1, claude-fable-5, claude-opus-5-5, claude-sonnet-5, claude-opus-4-8, claude-opus-4-7, claude-opus-4-6, claude-sonnet-4-6, claude-opus-4-5, claude-sonnet-4-5, claude-haiku-4-5.
• Google: Models such as gemini-3.7-flash, gemini-3.6-flash, gemini-3.5-flash, gemini-3.5-flash-lite, gemini-3.1-pro-preview, gemini-3.1-flash-lite, gemini-3-flash-preview, gemini-2.5-pro, gemini-2.5-flash, and gemini-2.5-flash-lite can be used.
• Mistral: Install with pip install "llm-api-adapter[mistral]". Available models are mistral-small-2603, mistral-medium-3-5, and mistral-large-2512; see the Mistral package README for Mistral-specific behaviour.
• xAI: Install with pip install "llm-api-adapter[xai]". Fixed model IDs are grok-4.7, grok-4.6, and grok-4.5; see the xAI package README for its capability matrix and data-handling notes.
• Qwen: Install with pip install "llm-api-adapter[qwen]". Fixed model IDs are qwen3.8-max, qwen3.8-flash, qwen3.7-plus, and qwen3.7-flash; every operation requires an explicit Frankfurt workspace_id. See the Qwen package README for its capability boundary, including unsupported PDF input.
• Kimi: Install with pip install "llm-api-adapter[kimi]". Fixed model IDs are kimi-k3 and kimi-k2.6; image bytes/data URIs are supported, while public image URLs and all PDF DocumentPart forms are rejected before HTTP. See the Kimi package README for reasoning, cache-pricing, and data-handling details.
• DeepSeek: Install with pip install "llm-api-adapter[deepseek]", or install llm-api-adapter-deepseek directly. The package exposes only the verified deepseek-flash model through the official Responses API, with text, tools, portable structured output, reasoning, streaming, and image URL/bytes/data-URI input. Documents, generic files, OCR, upload, and conversion are rejected before HTTP. See the DeepSeek package README for its compatibility matrix, continuation privacy, usage/cost boundary, and official references.
• Z.ai: Install with pip install "llm-api-adapter[zai]", or install llm-api-adapter-zai directly. The package exposes only verified glm-5.3-flash, with text, tools, reasoning, streaming, and image URL/bytes/data-URI input. Portable structured output and documents are rejected before HTTP. See the Z.ai package README for its capability matrix and request boundaries.

Example:

adapter = UniversalLLMAPIAdapter(
    organization="openai",
    model="gpt-5",
    api_key=openai_api_key
)

To switch to another provider, simply change the organization and model parameters.

Switching Providers

Here is an example of how to switch between different LLM providers using the SDK:

Note: Each instance of UniversalLLMAPIAdapter is tied to a specific provider and model. You cannot change the organization parameter for an existing adapter object. To use a different provider, you must create a new instance.

gpt = UniversalLLMAPIAdapter(
    organization="openai",
    model="gpt-5",
    api_key=openai_api_key
)
gpt_response = gpt.chat(messages=messages)
print(gpt_response.content)

claude = UniversalLLMAPIAdapter(
    organization="anthropic",
    model="claude-sonnet-4-5",
    api_key=anthropic_api_key
)
claude_response = claude.chat(messages=messages)
print(claude_response.content)

google = UniversalLLMAPIAdapter(
    organization="google",
    model="gemini-2.5-flash",
    api_key=google_api_key
)
google_response = google.chat(messages=messages)
print(google_response.content)

Example Use Case

Here is a comprehensive example that showcases all possible message types and interactions:

from llm_api_adapter.models.messages.chat_message import (
    AIMessage, Prompt, UserMessage
)                                               
from llm_api_adapter.universal_adapter import UniversalLLMAPIAdapter

messages = [
    Prompt(
        "You are a friendly assistant who explains complex concepts "
        "in simple terms."
    ),
    UserMessage("Hi! Can you explain how artificial intelligence works?"),
    AIMessage(
        "Sure! Artificial intelligence (AI) is a system that can perform "
        "tasks requiring human-like intelligence, such as recognizing images "
        "or understanding language. It learns by analyzing large amounts of "
        "data, finding patterns, and making predictions."
    ),
    UserMessage("How does AI learn?"),
]

adapter = UniversalLLMAPIAdapter(
    organization="openai",
    model="gpt-5",
    api_key=openai_api_key
)

response = adapter.chat(
    messages=messages,
    max_tokens=256,
    temperature=1.0,
    top_p=1.0
)
print(response.content)

The ChatResponse object returned by chat includes:

1. model: The model that generated the response.
2. response_id: Unique identifier for the response.
3. timestamp: Response generation time.
4. usage: Object containing input_tokens, output_tokens, and total_tokens.
5. currency: The currency used for cost calculation.
6. cost_input: Cost of input tokens.
7. cost_output: Cost of output tokens.
8. cost_total: Total combined cost.
9. cost_breakdown: Optional non-token CostLineItem values supplied by an organization package.
10. content: The generated text response.
11. finish_reason: Provider-native reason why generation stopped (for example, "stop" or "length").
12. refusal: Provider-normalized refusal detail, or None when the response was not a refusal.
13. incomplete_reason: Provider-normalized incomplete-generation reason, or None when generation completed.
14. parsed_json: Parsed JSON object for a valid completed structured result, otherwise None.
15. parsed_model: Typed Pydantic instance for a valid completed response_model result, otherwise None.

Timeout Support

The SDK supports per-request timeouts for all providers.

timeout_s parameter

timeout_s defines the maximum time (in seconds) the SDK will wait for an LLM response. It applies to chat(), achat(), stream_chat(), and astream_chat().

If the timeout is exceeded, the request is aborted and LLMAPITimeoutError is raised.

Example

chat_params = {
    "messages": messages,
    "timeout_s": 2.5
}
gpt = UniversalLLMAPIAdapter(
    organization="openai",
    model="gpt-5.2",
    api_key=openai_api_key
)
response = gpt.chat(**chat_params)

Notes

• Timeout is applied uniformly across all providers.
• The parameter is optional; if omitted, the provider default is used.
• Timeout affects the full request lifecycle (network + model execution).

Handling timeout errors

Timeouts raise a dedicated exception that can be handled explicitly:

from llm_api_adapter.errors import LLMAPITimeoutError

try:
    response = gpt.chat(**chat_params)
except LLMAPITimeoutError:
    # retry, fallback, or abort
    print("LLM request timed out")

Reasoning Support

This section describes the provider-neutral reasoning_level parameter for chat(), achat(), stream_chat(), and astream_chat(). It gives application code one input surface; it does not claim that every model exposes the same levels or consumes the same number of reasoning tokens.

response = adapter.chat(
    messages=[UserMessage("Solve this step-by-step")],
    reasoning_level=2048,
)

Default behavior

When reasoning_level is omitted, the adapter bypasses model-aware resolution and preserves the provider request behavior that existed before it. Pass a level, including "none", when the request must express reasoning intent.

reasoning_level parameter

reasoning_level is optional and provider-agnostic. The adapters translate it to the provider's native effort or thinking-budget format.

Supported forms:

• int — explicit reasoning-token budget
• str — one of the canonical levels: "none", "minimal", "low", "medium", "high", or "very_high"

For a verified categorical model, an exact provider value listed for that model is preserved. This permits native values such as "xhigh" or "max" where the model supports them, but those values are not portable. Other canonical strings are projected upward through the model's ordered native values; "none" becomes the model minimum with a warning if it cannot disable reasoning. An integer is treated as a fraction of the model context window, clamped to 0–100%, then rounded upward to an available categorical value. When the native list has no "none", its first value is the first positive step: "minimal" and a numeric percentage at or below that step resolve to it rather than skipping it.

For a verified numeric-budget model, canonical values from "minimal" through "very_high" are evenly interpolated from that model's documented minimum to maximum budget (with upward rounding). "none" resolves to zero when zero is supported, otherwise to the minimum with a warning. An integer is a literal budget: a value below the documented minimum falls back to that minimum with a warning, while a value above the documented maximum is forwarded so the provider can return its native validation error.

Named levels express relative intent. Do not treat "medium" or a numeric value as an identical token budget across providers or models.

Usage examples

# Named level
response = adapter.chat(
    messages=[UserMessage("Explain this")],
    reasoning_level="medium",
)

# Explicit numeric level
response = adapter.chat(
    messages=[UserMessage("Solve this step-by-step")],
    reasoning_level=2048,
)

# Explicitly disable reasoning where the model supports it
response = adapter.chat(
    messages=[UserMessage("Simple answer, no reasoning")],
    reasoning_level="none",
)

Provider independence

reasoning_level provides the same application-level entry point for all supported providers:

• same parameter name
• no provider-specific reasoning kwargs in application code
• provider-specific translation is handled inside the adapter

The available levels, native mapping, token budget, and whether reasoning can be disabled remain model/provider-dependent. This allows switching between OpenAI, Anthropic, and Google without rewriting the request shape, while keeping provider limitations explicit.

Provider references: OpenAI reasoning, Anthropic extended thinking, and Google thinking.

Tool / Function Calling

The SDK provides a unified provider‑agnostic tool calling interface.

Tools are defined using ToolSpec, and tool calls are returned in normalized form through ChatResponse.tool_calls.

The adapter does not execute tools. Tool execution must be implemented by the caller.

ToolSpec

from llm_api_adapter.models.tools import ToolSpec

tool = ToolSpec(
    name="get_weather",
    description="Get current weather for a city",
    json_schema={
        "type": "object",
        "properties": {
            "city": {"type": "string"}
        },
        "required": ["city"],
        "additionalProperties": False
    }
)

Tool parameters

All request methods support the same tool parameters:

• tools
• tool_choice
• parallel_tool_calls

Tool calls are returned as normalized ToolCall objects. The adapter does not execute them, so the caller appends each resulting ToolMessage and makes the follow-up request itself.

parallel_tool_calls is provider-dependent: Anthropic can enable or disable parallel tool use, OpenAI supports it for Chat Completions, and Google ignores it because Gemini has no equivalent option.

Tool round‑trip example

import json
from typing import Any, Dict

from llm_api_adapter.models.messages.chat_message import (
    Prompt,
    UserMessage,
    AIMessage,
    ToolMessage,
)
from llm_api_adapter.models.tools import ToolSpec
from llm_api_adapter.universal_adapter import UniversalLLMAPIAdapter


tools = [
    ToolSpec(
        name="get_weather",
        description="Get current weather for a city",
        json_schema={
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
            "additionalProperties": False,
        },
    )
]


def run_tool(name: str, args: Dict[str, Any]) -> Dict[str, Any]:
    if name == "get_weather":
        return {"city": args["city"], "temperature": 22, "unit": "C"}
    raise ValueError(f"Unknown tool: {name}")


adapter = UniversalLLMAPIAdapter(
    organization="openai",
    model="gpt-5.2",
    api_key=openai_api_key,
)

messages = [
    Prompt("If the user asks about weather, call get_weather."),
    UserMessage("What's the weather in Tel Aviv today?")
]

first = adapter.chat(
    messages=messages,
    tools=tools,
    tool_choice="auto",
    max_tokens=1000,
)

if first.tool_calls:
    messages.append(AIMessage(content="", tool_calls=first.tool_calls))

    for tc in first.tool_calls:
        result = run_tool(tc.name, tc.arguments)
        messages.append(
            ToolMessage(
                tool_call_id=tc.call_id,
                content=json.dumps(result)
            )
        )

    final = adapter.chat(
        messages=messages,
        previous_response=first,
        max_tokens=1000
    )
    print(final.content)

For a complete async tool round-trip, see the Async API guide.

previous_response parameter

previous_response accepts the ChatResponse returned by an earlier chat() or achat() call, and can also be passed to the streaming methods.

For OpenAI models that use the Responses API (o-series and newer GPT models), the adapter extracts response_id from the previous response and passes it to the API as previous_response_id. This enables stateful multi-turn conversations where the model retains context server-side, which reduces the tokens you need to send in subsequent turns.

For Anthropic, Google, and xAI, the parameter is accepted but ignored — context is carried entirely through the messages list regardless. The xAI package also does not expose its provider-specific store option; see its README for data-retention details.

DeepSeek is also stateless and does not send previous_response_id. Its package uses previous_response only to carry matching opaque reasoning replay metadata while the caller continues to send the complete messages history. That metadata is not rendered, serialized, represented, or logged; visible reasoning remains opt-in through capture_reasoning=True. See the DeepSeek continuation and privacy contract.

If you omit previous_response, the call works normally; you just won't get the stateful-session benefit on OpenAI Responses API models.

Structured Output

The SDK supports structured output from chat(), achat(), stream_chat(), and astream_chat() through a raw json_schema dict or a Pydantic model passed as response_model. The documented Core portable profile below is guaranteed across OpenAI, Anthropic, Google, Mistral, and xAI. Its explicit boundaries define the schema vocabulary that remains portable between organizations.

For streaming methods, parsed fields and terminal structured-output state are available on the finalized ChatResponse passed to on_done.

Core portable JSON Schema profile

The profile is the provider-neutral intersection enforced before an HTTP request. A portable schema has:

• an object as its root;
• only string, number, integer, boolean, object, and array types; nullable fields are expressed only as [<one type>, "null"];
• additionalProperties: false on every object, and a required list that contains every object property exactly once; use a nullable property instead of an optional one;
• a single object schema in items for every array;
• a non-empty enum when enum is used; and
• only the profile's structural and metadata keywords: type, properties, required, additionalProperties, items, enum, title, description, $id, and $schema.

Direct local references of the form #/$defs/<name> (including Pydantic-generated references) are inlined locally before this profile is checked. External, unresolved, recursive, or semantically combined $ref values are rejected with JSONSchemaError before a provider request. Other JSON Schema composition, validation, and conditional keywords are intentionally outside the portable profile.

OpenAI, Anthropic, and Google enforce this Core profile. The Mistral, xAI, Qwen, and Kimi organization packages enforce the same boundary; xAI additionally rejects the documented xAI-only invalid constructs before sending its request.

Pydantic Integration (response_model)

Pass a Pydantic BaseModel subclass as response_model — the adapter extracts the JSON Schema automatically and returns a typed instance in ChatResponse.parsed_model.

from pydantic import BaseModel, ConfigDict
from llm_api_adapter.models.messages.chat_message import Prompt, UserMessage
from llm_api_adapter.universal_adapter import UniversalLLMAPIAdapter

class Person(BaseModel):
    model_config = ConfigDict(extra="forbid")

    name: str
    age: int

adapter = UniversalLLMAPIAdapter(
    organization="openai",
    model="gpt-5",
    api_key=openai_api_key,
)

response = adapter.chat(
    messages=[
        Prompt("Extract structured data from the user's message."),
        UserMessage("My name is Alice and I'm 30 years old."),
    ],
    response_model=Person,
    max_tokens=200,
)

print(response.parsed_model)  # Person(name='Alice', age=30)
print(response.parsed_json)   # {"name": "Alice", "age": 30}

Note: Pydantic is not a required dependency of this package. Install it separately: pip install pydantic. Every nested Pydantic model must also use ConfigDict(extra="forbid") so its generated object schema satisfies the portable profile.

response_model cannot be combined with json_schema or tools — passing both raises JSONSchemaError.

Raw JSON Schema (json_schema)

The SDK also supports portable structured JSON output via a json_schema parameter in chat(), achat(), stream_chat(), and astream_chat(). The adapter preserves the schema's semantics: it validates the profile rather than silently adding required fields or changing additionalProperties.

from llm_api_adapter.models.messages.chat_message import Prompt, UserMessage
from llm_api_adapter.universal_adapter import UniversalLLMAPIAdapter

schema = {
    "type": "object",
    "properties": {
        "name": {"type": "string"},
        "age": {"type": "integer"},
    },
    "required": ["name", "age"],
    "additionalProperties": False,
}

adapter = UniversalLLMAPIAdapter(
    organization="openai",
    model="gpt-5",
    api_key=openai_api_key,
)

response = adapter.chat(
    messages=[
        Prompt("Extract structured data from the user's message."),
        UserMessage("My name is Alice and I'm 30 years old."),
    ],
    json_schema=schema,
    max_tokens=200,
)

print(response.content)      # '{"name": "Alice", "age": 30}'
print(response.parsed_json)  # {"name": "Alice", "age": 30}

json_schema parameter

• Accepts a dict containing a JSON Schema object.
• Cannot be combined with tools or response_model — passing both raises JSONSchemaError.
• Must satisfy the Core portable profile; unsupported, external, or recursive references raise JSONSchemaError before the request.
• Raw schemas are parsed as JSON only. The SDK does not run a local JSON Schema validator against the response.
• If a completed, non-refusal response is not valid JSON, JSONSchemaError is raised.

parsed_json field

ChatResponse.parsed_json contains the parsed dict for a valid completed structured result. It remains None for a refusal or incomplete generation, and parsed_model remains None in those cases as well.

Provider behavior

Provider	Implementation
OpenAI (standard)	Native response_format.type=json_schema with strict=true
OpenAI (Responses API / o-series)	Native text.format.type=json_schema with strict=true
Anthropic	Native output_config.format.type=json_schema
Google	generationConfig.responseMimeType="application/json" + responseJsonSchema
Mistral	Native response_format.type=json_schema with strict=true (optional organization package)
xAI	Responses text.format.type=json_schema with strict=true plus xAI's additive local overlay (optional organization package)
Qwen	Messages output_config.format.type=json_schema (optional organization package)

The same portable schema can be reused across these organizations without provider-specific request code. Google sends the portable JSON Schema through responseJsonSchema and drops only $id and $schema, which are non-semantic metadata; no transformer silently drops a semantic constraint.

Refusal, incomplete, and invalid results

A provider refusal is exposed as response.refusal; an incomplete generation is exposed as response.incomplete_reason. Neither is parsed or Pydantic validated, so both parsed fields remain None. A completed non-refusal result that is invalid JSON, or fails response_model.model_validate(), raises JSONSchemaError. Check the terminal fields before using structured output:

response = adapter.chat(
    messages=[UserMessage("Give me the data.")],
    json_schema=schema,
    max_tokens=200,
)
if response.refusal is not None:
    handle_refusal(response.refusal)
elif response.incomplete_reason is not None:
    handle_incomplete(response.incomplete_reason)
else:
    data = response.parsed_json

Error handling

from llm_api_adapter.errors import JSONSchemaError

try:
    response = adapter.chat(
        messages=[UserMessage("Give me the data.")],
        json_schema=schema,
        max_tokens=200,
    )
except JSONSchemaError as e:
    print(f"JSON schema error: {e}")

Vision Input

The SDK supports sending images alongside text using ImagePart and the files parameter on UserMessage. The input contract is shared across OpenAI, Anthropic, and Google; wire-format differences and provider-specific limitations are handled by the adapter.

Import

from llm_api_adapter.models.messages.chat_message import UserMessage
from llm_api_adapter.models.messages.file_parts import ImagePart

Image from URL

msg = UserMessage(
    "What is in this image?",
    files=[ImagePart(url="https://example.com/photo.jpg")]
)
response = adapter.chat(messages=[msg], max_tokens=200)
print(response.content)

Image from bytes

with open("photo.png", "rb") as f:
    image_bytes = f.read()

msg = UserMessage(
    "Describe this image.",
    files=[ImagePart(data=image_bytes, media_type="image/png")]
)
response = adapter.chat(messages=[msg], max_tokens=200)
print(response.content)

Multiple images

msg = UserMessage(
    "Compare these two images.",
    files=[
        ImagePart(url="https://example.com/before.jpg"),
        ImagePart(url="https://example.com/after.jpg"),
    ]
)

Supported formats

ImagePart accepts any image MIME type starting with image/ — image/jpeg, image/png, image/gif, image/webp, image/heic, image/heif.

When passing a URL, media_type is auto-detected from the file extension. For URLs without an extension, pass media_type explicitly:

ImagePart(url="https://api.example.com/image?id=42", media_type="image/jpeg")

OpenAI-style dict compatibility

The adapter also normalizes OpenAI-style content lists, so existing code that uses dicts works without changes:

messages = [{
    "role": "user",
    "content": [
        {"type": "text", "text": "What is this?"},
        {"type": "image_url", "image_url": {"url": "https://example.com/img.jpg"}},
    ]
}]
response = adapter.chat(messages=messages, max_tokens=200)

Kimi 0.1.0 accepts image bytes and data URIs for all three admitted models, but does not fetch public image URLs. ImagePart(url=...) is rejected before HTTP; use ImagePart(data=..., media_type="image/...") instead. See the Kimi package README.

DeepSeek deepseek-flash accepts user-message image URLs, bytes, and base64 data URIs for JPEG, PNG, GIF, and WebP images. Its adapter rejects unsupported image forms before HTTP and enforces the provider's URL, inline-size, and per-request image-count limits. See the DeepSeek image and file boundary and the official Vision guide.

Note: ImagePart is supported in v0.5.0; DocumentPart is introduced in v0.5.1. Google already supports audio input, but AudioPart is postponed because Anthropic does not support audio and OpenAI uses a separate audio API, so there is no common provider-neutral contract yet.

Document Input

The SDK supports PDF documents alongside text using DocumentPart and the files parameter on UserMessage when the selected organization supports document input. Provider-specific wire formats are handled automatically.

Import

from llm_api_adapter.models.messages.chat_message import UserMessage
from llm_api_adapter.models.messages.file_parts import DocumentPart

PDF from URL

msg = UserMessage(
    "Summarize this document in one sentence.",
    files=[DocumentPart(url="https://example.com/report.pdf")]
)
response = adapter.chat(messages=[msg], max_tokens=200)
print(response.content)

The PDF MIME type is detected from the .pdf extension. Anthropic, Google, and the OpenAI Responses API accept a document URL. OpenAI Chat Completions models below gpt-5 do not accept a document URL; use bytes instead.

PDF from bytes

with open("report.pdf", "rb") as f:
    pdf_bytes = f.read()

msg = UserMessage(
    "Summarize this document in one sentence.",
    files=[DocumentPart(data=pdf_bytes, media_type="application/pdf")]
)
response = adapter.chat(messages=[msg], max_tokens=200)
print(response.content)

For bytes, the adapter sends the PDF as base64 data in the provider-specific request format. The same DocumentPart works with Anthropic, Google, OpenAI Chat Completions, and the OpenAI Responses API.

File type support

File type	Anthropic	OpenAI (< gpt-5)	OpenAI (gpt-5+)	Google	Qwen	Kimi	DeepSeek
ImagePart (URL)	✅	✅	✅	✅	✅	❌	✅
ImagePart (bytes)	✅	✅	✅	✅	✅	✅	✅
DocumentPart (URL)	✅	❌	✅	✅	❌	❌	❌
DocumentPart (bytes)	✅	✅	✅	✅	❌	❌	❌

Qwen 0.1.0 supports images, but rejects every DocumentPart URL or byte before HTTP: PDF and OCR input are outside its package contract. See the Qwen package README.

Kimi 0.1.0 also rejects every DocumentPart URL or byte before HTTP. Kimi's Files API exposes extracted text rather than a Chat Completions attachment, so it cannot meet the same bytes-and-URL contract without hidden URL retrieval. The adapter does not upload or delete files for Kimi. See the Kimi package README.

DeepSeek 0.1.0 rejects every DocumentPart, generic non-image FilePart, OCR/upload/conversion route, and unsupported image form before either client is called. It does not fetch document URLs, upload files, process PDFs locally, or silently fall back to another endpoint or model. See the DeepSeek package file boundary.

Token Usage and Pricing

achat() returns the same usage, currency, and cost fields as chat(). For astream_chat(), the finalized response passed to on_done contains the same fields, while StreamChunk.usage is populated when the provider reports usage during streaming. cost_input and cost_output are reserved for token costs; non-token components use cost_breakdown.

Tiered standard-rate estimates

Costs are a standard-rate estimate, not a provider invoice. When a provider reports usage, the adapter selects the first pricing tier whose up_to_prompt_tokens is greater than or equal to usage.input_tokens. The boundary is inclusive; the final tier has no boundary. Its input and output rates are both applied to the entire request.

• context_window_tokens is the combined input and generated-output capacity.
• max_output_tokens is the generated-output capacity.
• A pricing-tier boundary only selects a price; it is not a request limit.
• If the provider does not report usage, the adapter does not estimate tokens locally. usage, currency, and cost_* remain None.

For providers with one static registry tier, the estimate covers bundled standard text input/output rates only. It excludes cached input, cache write/storage, batch, flex, priority, modality-specific, provider-hosted tool, and negotiated-volume charges. Do not use it to reconcile a provider invoice.

Kimi is the narrow exception when the provider reports an explicit usage.cached_tokens split: its organization package applies registered cache-hit and cache-miss input rates and can calculate cost_total. If Kimi does not return that split, cost_input and cost_total stay unset rather than assuming all input was uncached. This does not enable Kimi caching; it only accounts for provider-reported usage. See the Kimi package README.

DeepSeek deepseek-flash is another package-local exception. Its Responses usage reports input_tokens_details.cached_tokens and output_tokens_details.reasoning_tokens; the package applies the verified cache-hit/cache-miss and UTC peak/off-peak rates only when those values and the dispatch window are valid. Missing or inconsistent usage leaves the relevant cost fields unavailable. DeepSeek context caching is provider-managed and the estimate is not an invoice. See the DeepSeek usage and pricing boundary and the official pricing table.

Token Usage and Pricing Example

google = UniversalLLMAPIAdapter(
    organization="google",
    model="gemini-2.5-flash",
    api_key=google_api_key
)

response = google.chat(**chat_params)

if response.usage is None:
    print("Provider did not report usage; cost is unavailable.")
else:
    print(response.usage.input_tokens, "tokens", f"({response.cost_input} {response.currency})")
    print(response.usage.output_tokens, "tokens", f"({response.cost_output} {response.currency})")
    print(response.usage.total_tokens, "tokens", f"({response.cost_total} {response.currency})")

Prices are updated with each release to reflect provider changes. Bundled prices reflect each provider's standard API rates — batch pricing, cached-token discounts, and volume agreements are not accounted for. Use set_in_per_1m / set_out_per_1m to apply your actual rates if needed:

Overriding Pricing or Currency

An override replaces that rate in every pricing tier for the selected model. It affects token pricing only; it does not change registered metered-operation rates.

google = UniversalLLMAPIAdapter(
    organization="google",
    model="gemini-2.5-flash",
    api_key=google_api_key
)

google.pricing.set_in_per_1m(1.5)
google.pricing.set_out_per_1m(3)
google.pricing.set_currency("EUR")

response = google.chat(**chat_params)
print(response.content)
if response.usage is not None:
    print(response.usage.input_tokens, "tokens", f"({response.cost_input} {response.currency})")
    print(response.usage.output_tokens, "tokens", f"({response.cost_output} {response.currency})")
    print(response.usage.total_tokens, "tokens", f"({response.cost_total} {response.currency})")

Logging

The library uses Python's standard logging module and does not configure handlers. Loggers are module-based under llm_api_adapter.* (e.g., llm_api_adapter.universal_adapter).

• Default behavior: No handlers installed, effective level = WARNING.
• No secrets are logged — API keys and request bodies are excluded. Only event metadata and errors are logged. Adapter and client objects mask the key in __repr__ (e.g. api_key='sk-12345...cdef'), so they are safe to log or print.

Enable logs (console)

import logging

logging.basicConfig(level=logging.INFO)  # or DEBUG
# Optionally limit logging to this library
logging.getLogger("llm_api_adapter").setLevel(logging.DEBUG)

Write logs to a file

import logging

handler = logging.FileHandler("llm_api_adapter.log")
handler.setFormatter(logging.Formatter(
    "%(asctime)s %(levelname)s %(name)s %(message)s"
))
root = logging.getLogger()
root.setLevel(logging.INFO)
root.addHandler(handler)

Per-request correlation (optional)

import logging
logger = logging.getLogger("llm_api_adapter")

req_id = "req-123"
logger = logging.LoggerAdapter(logger, {"request_id": req_id})
logger.info("starting call")

To include request_id in log output, add %(request_id)s to your log formatter.

Reduce noise / silence logs

import logging
logging.getLogger("llm_api_adapter").setLevel(logging.WARNING)   # silence info logs
logging.getLogger("urllib3").setLevel(logging.WARNING)           # if using requests

Env-based log level toggle

# app.py
import logging, os
level = os.getenv("LLM_ADAPTER_LOGLEVEL", "WARNING").upper()
logging.getLogger("llm_api_adapter").setLevel(level)

Tip: For HTTP-level debugging with requests, also set:

import http.client as http_client, logging
http_client.HTTPConnection.debuglevel = 1
logging.getLogger("urllib3").setLevel(logging.DEBUG)

Use this only in development.

Adoption

Vertec bundles llm-api-adapter with Vertec Cloud Suite and On-Premises, allowing customers to call OpenAI, Anthropic, and Google models from custom Python extensions through a unified interface.

Related project

For production applications that need retries, multi-provider failover, circuit breakers, and tool-session recovery, see llm-api-resilience.

Install it with:

python -m pip install llm-api-resilience

llm-api-resilience is built on top of this adapter and keeps its provider-neutral interface unchanged.

Development & Testing

Developer setup, deterministic unit and mocked-integration commands, paid E2E requirements, provider-key safety, documentation rules, and the release flow are documented in CONTRIBUTING.md.

License

This project is licensed under the terms of the MIT License.
See the LICENSE file for details.