llm-client

ehs-llm-client is a unified async Python client for interacting with multiple LLM providers, including OpenAI, Azure OpenAI, and Google Gemini.
It supports single calls, structured JSON outputs, and batch processing, designed for production-ready, reusable code in applications, scripts, or pipelines.

Features

• Async support for multiple LLM providers
• Unified interface across OpenAI, Azure, and Google Gemini
• Structured JSON responses via schemas
• Retry and timeout handling
• Batch processing support
• Environment variable configuration for API keys
• Easy integration into existing projects or monorepos

Installation

# Install from PyPI
pip install ehs-llm-client

Or install editable version during development:

# From project root
pip install -e .

Configuration

ehs-llm-client supports three configuration modes:

1️⃣ Config file (.cfg)

Example: llmconfig.cfg

[default_settings]
provider = openai
model = gpt-4.1-mini

[prod]
provider = openai
model = gpt-4.1
model_batch = gpt-4.1-mini

Usage:

from ehs_llm_client import LLM

llm = LLM("prod", config_file_path="llmconfig.cfg")

2️⃣ Config dictionary (for tests / CI)

from ehs_llm_client import LLM

llm = LLM(
    "default",
    config={
        "default": {
            "provider": "openai",
            "model": "gpt-4.1-mini"
        }
    }
)

3️⃣ Environment variable fallback

export LLM_PROVIDER=openai
export LLM_MODEL=gpt-4.1-mini
export OPENAI_API_KEY=sk-...

llm = LLM("default")

Azure OpenAI with GPT-6

Deploy your chosen GPT-6 model in Azure, then use its deployment name as model:

[azure]
provider = azure_openai
model = my-gpt6-deployment
base_url = https://YOUR-RESOURCE.openai.azure.com/

Set AZURE_OPENAI_API_KEY and initialize the client:

llm = LLM("azure", config_file_path="llmconfig.cfg")

You can also configure everything with environment variables:

$env:LLM_PROVIDER = "azure_openai"
$env:LLM_MODEL = "my-gpt6-deployment"
$env:AZURE_OPENAI_ENDPOINT = "https://YOUR-RESOURCE.openai.azure.com/"
$env:AZURE_OPENAI_API_KEY = "YOUR-AZURE-API-KEY"

Then use LLM("default"). LLM_BASE_URL or a configured base_url takes precedence over AZURE_OPENAI_ENDPOINT. Both a resource URL and a full /openai/v1/ URL are accepted, including endpoints on services.ai.azure.com. The client uses Azure's v1 API; existing api_version settings are retained for compatibility and ignored. See Azure Responses API documentation for supported models and regions.

Usage

Single async call

import asyncio
from ehs_llm_client import LLM

async def main():
    llm = LLM("prod", config_file_path="llmconfig.cfg")
    messages = [{"role": "user", "content": "Say hello in JSON"}]

    response, in_tokens, out_tokens = await llm.get_response_async(
        messages, schema="return json"
    )

    print(response)

asyncio.run(main())

Blocked, failed or incomplete OpenAI/Azure responses raise ehs_llm_client.exceptions.LLMResponseError (a RuntimeError) instead of returning partial or empty text as success. Content-filter errors are not retried with the same prompt.

Reasoning effort

The reasoning controls from version 0.1.10 are supported:

[reasoning]
provider = openai
model = gpt-6-astra
default_reasoning_effort = medium

Set default_reasoning_effort in a config section/dictionary, or set LLM_DEFAULT_REASONING_EFFORT when using environment configuration. Override it for a single call with reasoning_effort:

response, in_tokens, out_tokens = await llm.get_response_async(
    messages, schema="return json", reasoning_effort="high"
)

call_async also accepts reasoning_effort. OpenAI uses the model support check from 0.1.10. Azure forwards an explicitly configured effort because deployment names can be arbitrary; only set it for a reasoning-capable deployment. Supported effort values depend on the model. If you omit the setting, the API's default applies. See OpenAI reasoning guidance. The setting applies to single calls; batch behavior remains as in 0.1.10.

Use keyword arguments for retry, timeout and reasoning settings. Both older positional calls (messages, schema, max_retries, timeout) and 0.1.10-style calls (messages, schema, reasoning_effort, max_retries, timeout) remain supported.

OpenAI/Azure JSON mode (schema="return json") requires string message contents, as in 0.1.10. Regular calls and schema-based output retain support for content arrays.

Batch request

batch_line = llm.create_batch_request(
    custom_id="test1",
    messages=[{"role": "user", "content": "Give me a JSON object"}],
    schema="return json"
)

batch_id = await llm.run_batch_process([batch_line], submission_id="batch123")
status = await llm.get_batch_status(batch_id)
print(status)

create_batch_request returns a dictionary. run_batch_process accepts these dictionaries or JSONL strings that you already serialized. When model_batch is omitted, OpenAI/Azure batches use model.

For Azure, set model_batch to a deployment that supports batch processing (GlobalBatch or DataZoneBatch). Model and region availability can differ from single-call deployments; see Azure batch documentation. Azure supports submission, status checking, cancellation and result downloads through the same client methods.

API Key Management

Keep API keys out of code. Recommended environment variables:

The original APIkey spelling takes precedence, followed by 0.1.10's APIKey spelling and then the standard variable. Azure credentials must come from your Azure resource.

Development

# Create a virtual environment
python -m venv .venv
source .venv/bin/activate  # macOS/Linux
.venv\Scripts\activate     # Windows

# Install dependencies
pip install -e .

Run the regression checks (simulated HTTP responses; no API credentials needed):

python -m unittest discover -s tests -v

License

MIT License © 2026 Your Name