Skip to main content

llm-client

PyPI version Python Version

ehs-llm-client is a unified async Python client for interacting with multiple LLM providers, including OpenAI, Azure OpenAI, and Google Gemini.
It supports single calls, structured JSON outputs, and batch processing, designed for production-ready, reusable code in applications, scripts, or pipelines.


Features

  • Async support for multiple LLM providers
  • Unified interface across OpenAI, Azure, and Google Gemini
  • Structured JSON responses via schemas
  • Retry and timeout handling
  • Batch processing support
  • Environment variable configuration for API keys
  • Easy integration into existing projects or monorepos

Installation

# Install from PyPI
pip install ehs-llm-client

Or install editable version during development:

# From project root
pip install -e .

Configuration

ehs-llm-client supports three configuration modes:

1️⃣ Config file (.cfg)

Example: llmconfig.cfg

[default_settings]
provider = openai
model = gpt-4.1-mini

[prod]
provider = openai
model = gpt-4.1
model_batch = gpt-4.1-mini

Usage:

from ehs_llm_client import LLM

llm = LLM("prod", config_file_path="llmconfig.cfg")

2️⃣ Config dictionary (for tests / CI)

from ehs_llm_client import LLM

llm = LLM(
    "default",
    config={
        "default": {
            "provider": "openai",
            "model": "gpt-4.1-mini"
        }
    }
)

3️⃣ Environment variable fallback

export LLM_PROVIDER=openai
export LLM_MODEL=gpt-4.1-mini
export OPENAI_API_KEY=sk-...
llm = LLM("default")

Azure OpenAI with GPT-6

Deploy your chosen GPT-6 model in Azure, then use its deployment name as model:

[azure]
provider = azure_openai
model = my-gpt6-deployment
base_url = https://YOUR-RESOURCE.openai.azure.com/

Set AZURE_OPENAI_API_KEY and initialize the client:

llm = LLM("azure", config_file_path="llmconfig.cfg")

You can also configure everything with environment variables:

$env:LLM_PROVIDER = "azure_openai"
$env:LLM_MODEL = "my-gpt6-deployment"
$env:AZURE_OPENAI_ENDPOINT = "https://YOUR-RESOURCE.openai.azure.com/"
$env:AZURE_OPENAI_API_KEY = "YOUR-AZURE-API-KEY"

Then use LLM("default"). LLM_BASE_URL or a configured base_url takes precedence over AZURE_OPENAI_ENDPOINT. Both a resource URL and a full /openai/v1/ URL are accepted, including endpoints on services.ai.azure.com. The client uses Azure's v1 API; existing api_version settings are retained for compatibility and ignored. See Azure Responses API documentation for supported models and regions.


Usage

Single async call

import asyncio
from ehs_llm_client import LLM

async def main():
    llm = LLM("prod", config_file_path="llmconfig.cfg")
    messages = [{"role": "user", "content": "Say hello in JSON"}]

    response, in_tokens, out_tokens = await llm.get_response_async(
        messages, schema="return json"
    )

    print(response)

asyncio.run(main())

One LLM can be used from several event loops in turn, for example a Celery task that calls asyncio.run() for each group of work. Since 0.1.13, llm.client is replaced with a new provider client the first time it is used from a new event loop. Before that, calls in a later loop reused connections opened by the closed one; they failed, and the OpenAI SDK retried them ("Retrying request to /responses"). Assigning llm.client yourself still works; the next event loop to use it adopts it.

Blocked, failed or incomplete OpenAI/Azure responses raise ehs_llm_client.exceptions.LLMResponseError (a RuntimeError) instead of returning partial or empty text as success. Content-filter errors are not retried with the same prompt.

Reasoning effort

The reasoning controls from version 0.1.10 are supported:

[reasoning]
provider = openai
model = gpt-6-astra
default_reasoning_effort = medium

Set default_reasoning_effort in a config section/dictionary, or set LLM_DEFAULT_REASONING_EFFORT when using environment configuration. Override it for a single call with reasoning_effort:

response, in_tokens, out_tokens = await llm.get_response_async(
    messages, schema="return json", reasoning_effort="high"
)

call_async also accepts reasoning_effort. OpenAI uses the model support check from 0.1.10. Azure forwards an explicitly configured effort because deployment names can be arbitrary; only set it for a reasoning-capable deployment. Supported effort values depend on the model. If you omit the setting, the API's default applies. See OpenAI reasoning guidance. The setting applies to single calls; batch behavior remains as in 0.1.10.

The 0.1.10 method signatures are preserved, including positional reasoning, retry and timeout arguments. Prefer keyword arguments for clarity.

OpenAI/Azure JSON mode (schema="return json") requires string message contents, as in 0.1.10. Regular calls and schema-based output retain support for content arrays.


Batch request

batch_line = llm.create_batch_request(
    custom_id="test1",
    messages=[{"role": "user", "content": "Give me a JSON object"}],
    schema="return json"
)

batch_id = await llm.run_batch_process([batch_line], submission_id="batch123")
status = await llm.get_batch_status(batch_id)
print(status)

create_batch_request returns a dictionary. run_batch_process accepts these dictionaries or JSONL strings that you already serialized. When model_batch is omitted, OpenAI/Azure batches use model.

For Azure, set model_batch to a deployment that supports batch processing (GlobalBatch or DataZoneBatch). Model and region availability can differ from single-call deployments; see Azure batch documentation. Azure supports submission, status checking, cancellation and result downloads through the same client methods.


API Key Management

Keep API keys out of code. Recommended environment variables:

Provider Standard env var Existing env vars (still supported)
OpenAI OPENAI_API_KEY openaiAPIkey, openaiAPIKey
Azure OpenAI AZURE_OPENAI_API_KEY azure_openaiAPIkey, azure_openaiAPIKey
Google Gemini GOOGLE_API_KEY googleAPIkey, googleAPIKey

The 0.1.10 APIKey spelling takes precedence, followed by the older APIkey spelling and then the standard variable. Azure credentials must come from your Azure resource.


Development

# Create a virtual environment
python -m venv .venv
source .venv/bin/activate  # macOS/Linux
.venv\Scripts\activate     # Windows

# Install dependencies
pip install -e .

Run the regression checks (simulated HTTP responses; no API credentials needed):

python -m unittest discover -s tests -v

License

MIT License © 2026 Your Name

Metadata

Release files for ehs-llm-client 0.1.13

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for ehs-llm-client 0.1.13
File Size Uploaded Provenance Actions
ehs_llm_client-0.1.13.tar.gz 18.2 kB Without provenance Details for ehs_llm_client-0.1.13.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for ehs-llm-client 0.1.13
File Interpreter ABI Platform Actions
ehs_llm_client-0.1.13-py3-none-any.whl Python 3 none any Details for ehs_llm_client-0.1.13-py3-none-any.whl

Total release size: 29.8 kB

Release files / ehs_llm_client-0.1.13.tar.gz

Download URL ehs_llm_client-0.1.13.tar.gz
Size 18.2 kB
Tags Source
SHA-256 checksum
How to use checksums
864602603e4241937a0098a7c22987a2982b93a84fa8193cd85547698856472a
BLAKE2b-256 checksum
How to use checksums
94e243a4f21f6314643bc36a3fdf504607b9b1147b4116e4206ec21e15746be5
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.13.5

Release files / ehs_llm_client-0.1.13-py3-none-any.whl

Download URL ehs_llm_client-0.1.13-py3-none-any.whl
Size 11.5 kB
Tags Python 3
SHA-256 checksum
How to use checksums
752084ab0ba4099a6f9a99ffc66e8127a365ed20c749f42144eb9d78c3aac1c1
BLAKE2b-256 checksum
How to use checksums
491326e794665f34010828a1c235767c2aacdcf30c76fe6d413b5d095e54c2da
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.13.5

Security for ehs-llm-client 0.1.13

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page