llm-client
ehs-llm-client is a unified async Python client for interacting with multiple LLM providers, including OpenAI, Azure OpenAI, and Google Gemini.
It supports single calls, structured JSON outputs, and batch processing, designed for production-ready, reusable code in applications, scripts, or pipelines.
Features
- Async support for multiple LLM providers
- Unified interface across OpenAI, Azure, and Google Gemini
- Structured JSON responses via schemas
- Retry and timeout handling
- Batch processing support
- Environment variable configuration for API keys
- Easy integration into existing projects or monorepos
Installation
# Install from PyPI
pip install ehs-llm-client
Or install editable version during development:
# From project root
pip install -e .
Configuration
ehs-llm-client supports three configuration modes:
1️⃣ Config file (.cfg)
Example: llmconfig.cfg
[default_settings]
provider = openai
model = gpt-4.1-mini
[prod]
provider = openai
model = gpt-4.1
model_batch = gpt-4.1-mini
Usage:
from ehs_llm_client import LLM
llm = LLM("prod", config_file_path="llmconfig.cfg")
2️⃣ Config dictionary (for tests / CI)
from ehs_llm_client import LLM
llm = LLM(
"default",
config={
"default": {
"provider": "openai",
"model": "gpt-4.1-mini"
}
}
)
3️⃣ Environment variable fallback
export LLM_PROVIDER=openai
export LLM_MODEL=gpt-4.1-mini
export OPENAI_API_KEY=sk-...
llm = LLM("default")
Azure OpenAI with GPT-6
Deploy your chosen GPT-6 model in Azure, then use its deployment name as model:
[azure]
provider = azure_openai
model = my-gpt6-deployment
base_url = https://YOUR-RESOURCE.openai.azure.com/
Set AZURE_OPENAI_API_KEY and initialize the client:
llm = LLM("azure", config_file_path="llmconfig.cfg")
You can also configure everything with environment variables:
$env:LLM_PROVIDER = "azure_openai"
$env:LLM_MODEL = "my-gpt6-deployment"
$env:AZURE_OPENAI_ENDPOINT = "https://YOUR-RESOURCE.openai.azure.com/"
$env:AZURE_OPENAI_API_KEY = "YOUR-AZURE-API-KEY"
Then use LLM("default"). LLM_BASE_URL or a configured base_url takes precedence
over AZURE_OPENAI_ENDPOINT. Both a resource URL and a full /openai/v1/ URL are
accepted, including endpoints on services.ai.azure.com. The client uses Azure's
v1 API; existing api_version settings are retained for compatibility and ignored.
See Azure Responses API documentation
for supported models and regions.
Usage
Single async call
import asyncio
from ehs_llm_client import LLM
async def main():
llm = LLM("prod", config_file_path="llmconfig.cfg")
messages = [{"role": "user", "content": "Say hello in JSON"}]
response, in_tokens, out_tokens = await llm.get_response_async(
messages, schema="return json"
)
print(response)
asyncio.run(main())
One LLM can be used from several event loops in turn, for example a Celery task
that calls asyncio.run() for each group of work. Since 0.1.13, llm.client is
replaced with a new provider client the first time it is used from a new event
loop. Before that, calls in a later loop reused connections opened by the closed
one; they failed, and the OpenAI SDK retried them ("Retrying request to
/responses"). Assigning llm.client yourself still works; the next event loop
to use it adopts it.
Connections and concurrency
Hosts such as Azure App Service give each instance a limited number of outbound
(SNAT) ports per destination, about 128. Every open connection uses one. When
they run out, new connections time out (httpx.ConnectTimeout, then
openai.APITimeoutError) while existing ones keep working. Three settings,
read from the environment, control how many connections the client opens:
| Variable | Default | Effect |
|---|---|---|
LLM_MAX_CONCURRENCY |
0 (no limit) |
Most get_response_async requests in flight at once, shared by every LLM in the process (per event loop). Further calls wait for a slot. Waiting does not count toward timeout, and a call holds no slot while it sleeps between retries. Since 0.1.15. |
LLM_HTTP2 |
off | true sends OpenAI and Azure OpenAI requests over HTTP/2, so concurrent requests share connections instead of opening one each. Servers without HTTP/2 are spoken to over HTTP/1.1. Since 0.1.15. |
LLM_KEEPALIVE_EXPIRY |
60 |
Seconds an idle connection is kept for reuse (the OpenAI SDK default is 5), so calls that come in groups reuse connections instead of opening new ones. Must be more than 0 and less than 240: Azure drops outbound connections idle for longer without telling either end. Since 0.1.14. |
The ports a process can use at once are about LLM_MAX_CONCURRENCY over
HTTP/1.1, and far fewer over HTTP/2. Several processes on one instance share
its ports, so size the limit as the instance's budget divided by the number of
processes that make calls. A lower limit means calls in a large burst wait for
each other; a burst that is never larger than the limit is not slowed.
HTTP/2 trade-offs: a connection that drops or loses packets affects every request on it at once, and httpx's HTTP/2 support has seen less use than its HTTP/1.1 support. Turn it on in a test environment first.
Other SDK defaults are unchanged: connect timeout 5 s, read timeout 600 s, at
most 1000 connections per client. Batch file and job calls are not limited by
LLM_MAX_CONCURRENCY.
Blocked, failed or incomplete OpenAI/Azure responses raise
ehs_llm_client.exceptions.LLMResponseError (a RuntimeError) instead of returning
partial or empty text as success. Content-filter errors are not retried with the same prompt.
Reasoning effort
The reasoning controls from version 0.1.10 are supported:
[reasoning]
provider = openai
model = gpt-6-astra
default_reasoning_effort = medium
Set default_reasoning_effort in a config section/dictionary, or set
LLM_DEFAULT_REASONING_EFFORT when using environment configuration. Override it
for a single call with reasoning_effort:
response, in_tokens, out_tokens = await llm.get_response_async(
messages, schema="return json", reasoning_effort="high"
)
call_async also accepts reasoning_effort. OpenAI uses the model support check
from 0.1.10. Azure forwards an explicitly configured effort because deployment
names can be arbitrary; only set it for a reasoning-capable deployment. Supported
effort values depend on the model. If you omit the setting, the API's default applies.
See OpenAI reasoning guidance.
The setting applies to single calls; batch behavior remains as in 0.1.10.
The 0.1.10 method signatures are preserved, including positional reasoning, retry and timeout arguments. Prefer keyword arguments for clarity.
OpenAI/Azure JSON mode (schema="return json") requires string message contents,
as in 0.1.10. Regular calls and schema-based output retain support for content arrays.
Batch request
batch_line = llm.create_batch_request(
custom_id="test1",
messages=[{"role": "user", "content": "Give me a JSON object"}],
schema="return json"
)
batch_id = await llm.run_batch_process([batch_line], submission_id="batch123")
status = await llm.get_batch_status(batch_id)
print(status)
create_batch_request returns a dictionary. run_batch_process accepts these
dictionaries or JSONL strings that you already serialized. When model_batch is
omitted, OpenAI/Azure batches use model.
For Azure, set model_batch to a deployment that supports batch processing
(GlobalBatch or DataZoneBatch). Model and region availability can differ from
single-call deployments; see Azure batch documentation.
Azure supports submission, status checking, cancellation and result downloads
through the same client methods.
API Key Management
Keep API keys out of code. Recommended environment variables:
| Provider | Standard env var | Existing env vars (still supported) |
|---|---|---|
| OpenAI | OPENAI_API_KEY |
openaiAPIkey, openaiAPIKey |
| Azure OpenAI | AZURE_OPENAI_API_KEY |
azure_openaiAPIkey, azure_openaiAPIKey |
| Google Gemini | GOOGLE_API_KEY |
googleAPIkey, googleAPIKey |
The 0.1.10 APIKey spelling takes precedence, followed by the older APIkey
spelling and then the standard variable. Azure credentials must come from your Azure resource.
Development
# Create a virtual environment
python -m venv .venv
source .venv/bin/activate # macOS/Linux
.venv\Scripts\activate # Windows
# Install dependencies
pip install -e .
Run the regression checks (simulated HTTP responses; no API credentials needed):
python -m unittest discover -s tests -v
License
MIT License © 2026 Your Name
Metadata
Release files for ehs-llm-client 0.1.15
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| ehs_llm_client-0.1.15.tar.gz | 22.0 kB | Without provenance | Details for ehs_llm_client-0.1.15.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| ehs_llm_client-0.1.15-py3-none-any.whl | Python 3 | none | any | Details for ehs_llm_client-0.1.15-py3-none-any.whl |
Total release size: 35.4 kB
Release files / ehs_llm_client-0.1.15.tar.gz
| Download URL | ehs_llm_client-0.1.15.tar.gz |
|---|---|
| Size | 22.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
453962c2e3d55021af59fef4d113efd0541b00dd0dda2245fad37ede66494079
|
|
BLAKE2b-256 checksum How to use checksums |
7a38c909b253c3613ff9870907326c71bffce9b09a918eba5f673a5770f2446e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.13.5
|
Release files / ehs_llm_client-0.1.15-py3-none-any.whl
| Download URL | ehs_llm_client-0.1.15-py3-none-any.whl |
|---|---|
| Size | 13.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
5cc857824372b89c79044f7134d9d9e3819d894e359d543a508436035645127a
|
|
BLAKE2b-256 checksum How to use checksums |
8f5ec38ab2d46c10aa66503154e8a34352b732460903182e842983e105d5a594
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.13.5
|
Security for ehs-llm-client 0.1.15
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.