Skip to main content

llm-client

PyPI version Python Version

ehs-llm-client is a unified async Python client for OpenAI, Azure OpenAI, Microsoft Foundry (AIF), Google Gemini, and TypeSafe/Jev. It supports generative responses and native typed decisions. Provider batch APIs remain available for supported generative deployments.


Features

  • Async support for multiple LLM providers
  • Unified interface across OpenAI, Azure, and Google Gemini
  • Structured JSON responses via schemas
  • Native choice, noul and score decisions via Foundry and TypeSafe
  • Retry and timeout handling
  • Batch processing support
  • Environment variable configuration for API keys
  • Easy integration into existing projects or monorepos

Installation

# Install from PyPI
pip install ehs-llm-client

Or install editable version during development:

# From project root
pip install -e .

Configuration

ehs-llm-client supports three configuration modes:

1️⃣ Config file (.cfg)

Example: llmconfig.cfg

[default_settings]
provider = openai
model = gpt-4.1-mini

[prod]
provider = openai
model = gpt-4.1
model_batch = gpt-4.1-mini

Usage:

from ehs_llm_client import LLM

llm = LLM("prod", config_file_path="llmconfig.cfg")

2️⃣ Config dictionary (for tests / CI)

from ehs_llm_client import LLM

llm = LLM(
    "default",
    config={
        "default": {
            "provider": "openai",
            "model": "gpt-4.1-mini"
        }
    }
)

3️⃣ Environment variable fallback

export LLM_PROVIDER=openai
export LLM_MODEL=gpt-4.1-mini
export OPENAI_API_KEY=sk-...
llm = LLM("default")

Azure OpenAI with GPT-6

Deploy your chosen GPT-6 model in Azure, then use its deployment name as model:

[azure]
provider = azure_openai
model = my-gpt6-deployment
base_url = https://YOUR-RESOURCE.openai.azure.com/

Set AZURE_OPENAI_API_KEY and initialize the client:

llm = LLM("azure", config_file_path="llmconfig.cfg")

You can also configure everything with environment variables:

$env:LLM_PROVIDER = "azure_openai"
$env:LLM_MODEL = "my-gpt6-deployment"
$env:AZURE_OPENAI_ENDPOINT = "https://YOUR-RESOURCE.openai.azure.com/"
$env:AZURE_OPENAI_API_KEY = "YOUR-AZURE-API-KEY"

Then use LLM("default"). LLM_BASE_URL or a configured base_url takes precedence over AZURE_OPENAI_ENDPOINT. Both a resource URL and a full /openai/v1/ URL are accepted, including endpoints on services.ai.azure.com. The client uses Azure's v1 API; existing api_version settings are retained for compatibility and ignored. See Azure Responses API documentation for supported models and regions.

Microsoft Foundry (AIF) and TypeSafe

Version 0.2.0 adds two providers. Existing providers, credential precedence and the get_response_async() positional arguments and return tuple are preserved.

Provider api Method Endpoint
azure_foundry responses (default) get_response_async() /openai/v1/responses
azure_foundry systemone decide_async() /providers/microsoft/v1/systemone
typesafe systemone (default) decide_async() https://api.typesafe.ai/v1/systemone

Select the API explicitly for Foundry decision deployments; deployment names do not determine the protocol. This release supports these two APIs, rather than every model/API in the Foundry catalog.

[foundry_gpt]
provider = azure_foundry
api = responses
model = gpt-6-luna
base_url = https://YOUR-RESOURCE.services.ai.azure.com
default_reasoning_effort = low

[foundry_decision]
provider = azure_foundry
api = systemone
model = decision-1
base_url = https://YOUR-RESOURCE.services.ai.azure.com

[typesafe_jev]
provider = typesafe
model = jev-latest

For Foundry, model is the deployment name. For TypeSafe, it is the model name or alias. Foundry uses its own AZURE_FOUNDRY_API_KEY and AZURE_FOUNDRY_ENDPOINT; it never falls back to Azure OpenAI credentials. A configured base_url takes precedence over LLM_BASE_URL, then the provider-specific endpoint variable. TypeSafe defaults to https://api.typesafe.ai; an explicit base_url or environment-only LLM_BASE_URL can override it.

Foundry expects a resource endpoint, not an /api/projects/... URL. Responses also accepts a full /openai/v1/ endpoint. System One accepts the resource root, its corresponding versioned API prefix, or its full System One endpoint. These endpoints must use HTTPS and contain no query string, fragment or embedded credentials. Selecting azure_foundry does not itself establish project billing attribution; that depends on the Azure endpoint and service capabilities.

For environment-only configuration, set LLM_PROVIDER, LLM_MODEL, and LLM_API=systemone for Foundry decision models, together with its endpoint/key. TypeSafe needs LLM_PROVIDER=typesafe, LLM_MODEL=jev-latest, and TYPESAFE_API_KEY.

See the Microsoft Decision API and TypeSafe System One API.


Usage

Single async call

import asyncio
from ehs_llm_client import LLM

async def main():
    llm = LLM("prod", config_file_path="llmconfig.cfg")
    messages = [{"role": "user", "content": "Say hello in JSON"}]

    response, in_tokens, out_tokens = await llm.get_response_async(
        messages, schema="return json"
    )

    print(response)

asyncio.run(main())

One LLM can be used from several event loops in turn, for example a Celery task that calls asyncio.run() for each group of work. Since 0.1.13, llm.client is replaced with a new provider client the first time it is used from a new event loop. Before that, calls in a later loop reused connections opened by the closed one; they failed, and the OpenAI SDK retried them ("Retrying request to /responses"). Assigning llm.client yourself still works; the next event loop to use it adopts it.

Connections and concurrency

Hosts such as Azure App Service give each instance a limited number of outbound (SNAT) ports per destination, about 128. Every open connection uses one. When they run out, new connections time out (httpx.ConnectTimeout, then openai.APITimeoutError) while existing ones keep working. Three settings, read from the environment, control how many connections the client opens:

Variable Default Effect
LLM_MAX_CONCURRENCY 0 (no limit) Most get_response_async requests in flight at once, shared by every LLM in the process (per event loop). Further calls wait for a slot. Waiting does not count toward timeout, and a call holds no slot while it sleeps between retries. Since 0.1.15.
LLM_HTTP2 off true sends OpenAI and Azure OpenAI requests over HTTP/2, so concurrent requests share connections instead of opening one each. Servers without HTTP/2 are spoken to over HTTP/1.1. Since 0.1.15.
LLM_KEEPALIVE_EXPIRY 60 Seconds an idle connection is kept for reuse (the OpenAI SDK default is 5), so calls that come in groups reuse connections instead of opening new ones. Must be more than 0 and less than 240: Azure drops outbound connections idle for longer without telling either end. Since 0.1.14.

The ports a process can use at once are about LLM_MAX_CONCURRENCY over HTTP/1.1, and far fewer over HTTP/2. Several processes on one instance share its ports, so size the limit as the instance's budget divided by the number of processes that make calls. A lower limit means calls in a large burst wait for each other; a burst that is never larger than the limit is not slowed.

HTTP/2 trade-offs: a connection that drops or loses packets affects every request on it at once, and httpx's HTTP/2 support has seen less use than its HTTP/1.1 support. Turn it on in a test environment first.

Other SDK defaults are unchanged: connect timeout 5 s, read timeout 600 s, at most 1000 connections per client. Batch file and job calls are not limited by LLM_MAX_CONCURRENCY.

Blocked, failed or incomplete OpenAI/Azure responses raise ehs_llm_client.exceptions.LLMResponseError (a RuntimeError) instead of returning partial or empty text as success. Content-filter errors are not retried with the same prompt.

Reasoning effort

The reasoning controls from version 0.1.10 are supported:

[reasoning]
provider = openai
model = gpt-6-astra
default_reasoning_effort = medium

Set default_reasoning_effort in a config section/dictionary, or set LLM_DEFAULT_REASONING_EFFORT when using environment configuration. Override it for a single call with reasoning_effort:

response, in_tokens, out_tokens = await llm.get_response_async(
    messages, schema="return json", reasoning_effort="high"
)

call_async also accepts reasoning_effort. OpenAI uses the model support check from 0.1.10. Azure forwards an explicitly configured effort because deployment names can be arbitrary; only set it for a reasoning-capable deployment. Supported effort values depend on the model. If you omit the setting, the API's default applies. See OpenAI reasoning guidance. The setting applies to single calls; batch behavior remains as in 0.1.10.

The 0.1.10 method signatures are preserved, including positional reasoning, retry and timeout arguments. Prefer keyword arguments for clarity.

OpenAI/Azure JSON mode (schema="return json") requires string message contents, as in 0.1.10. Regular calls and schema-based output retain support for content arrays.

Native typed decisions

Use the same questions with either a Foundry systemone section or a TypeSafe section. State can be text, a JSON object or an array. Instructions and criteria descriptions may also contain JSON structures.

import asyncio
from ehs_llm_client import LLM, ChoiceAnswer

async def main():
    async with LLM("typesafe_jev", config_file_path="llmconfig.cfg") as model:
        result = await model.decide_async(
            state="A worker suffered a minor cut and received first aid.",
            questions={
                "category": {
                    "type": "choice",
                    "instructions": "Classify the reported incident.",
                    "criteria": {
                        "injury": "An injury occurred.",
                        "near_miss": "An event occurred without injury.",
                        "unknown": "There is insufficient information.",
                    },
                },
                "injured": {"type": "noul", "instructions": "Was anyone injured?"},
                "severity": {
                    "type": "score",
                    "instructions": "Rate the reported outcome.",
                    "criteria": ["No injury", "Minor injury", "Serious injury"],
                },
            },
            max_retries=3,
            timeout=30,
        )
        answer = result.answers["category"]
        if isinstance(answer, ChoiceAnswer):
            print(answer.choice, answer.probabilities, answer.confidence)
        print(result.model, result.usage.input_tokens, result.usage.output_tokens)

asyncio.run(main())

DecisionResult contains typed ChoiceAnswer, NoulAnswer and ScoreAnswer objects, a resolved model, DecisionUsage, an optional request_id, and the complete provider JSON in raw_response. Choice/score probabilities, confidence and score legends are preserved when supplied; absent optional fields are None. Noul is a probability from 0 to 1, not a boolean. Score is a number between zero and the last criterion's index and can fall between levels. The library does not choose thresholds, turn uncertainty into labels, or translate answers for EIDA.

Input and answer shapes, allowed labels, numeric ranges and token usage are validated. Malformed successful responses raise LLMResponseError. HTTP failures raise LLMAPIError (a subclass) with status_code and optional request_id; error messages exclude provider response bodies. Inspect raw_response carefully before logging it because provider extensions can contain sensitive information.

Decision calls retry transport failures, timeouts and HTTP 408, 429, 500, 502, 503, 504 and 529, honoring Retry-After and retry-after-ms when valid. Other HTTP failures and invalid answers are not retried. As with the existing library API, max_retries means total attempts, including the first request; 1 disables retries. timeout is the limit for each attempt, so retries and backoff can make the complete call longer. Cancellation propagates to the caller.

System One clients reject generative calls, provider batch operations, model_batch, and default_reasoning_effort. Use concurrent decide_async() calls for application-level parallelism. Generative clients reject decide_async().

All providers support async with LLM(...) and await llm.aclose() to release their clients. Keep a client open across repeated calls to reuse connections.


Batch request

batch_line = llm.create_batch_request(
    custom_id="test1",
    messages=[{"role": "user", "content": "Give me a JSON object"}],
    schema="return json"
)

batch_id = await llm.run_batch_process([batch_line], submission_id="batch123")
status = await llm.get_batch_status(batch_id)
print(status)

create_batch_request returns a dictionary. run_batch_process accepts these dictionaries or JSONL strings that you already serialized. When model_batch is omitted, OpenAI/Azure batches use model.

For Azure, set model_batch to a deployment that supports batch processing (GlobalBatch or DataZoneBatch). Model and region availability can differ from single-call deployments; see Azure batch documentation. Azure supports submission, status checking, cancellation and result downloads through the same client methods.


API Key Management

Keep API keys out of code. Recommended environment variables:

Provider Standard env var Existing env vars (still supported)
OpenAI OPENAI_API_KEY openaiAPIkey, openaiAPIKey
Azure OpenAI AZURE_OPENAI_API_KEY azure_openaiAPIkey, azure_openaiAPIKey
Google Gemini GOOGLE_API_KEY googleAPIkey, googleAPIKey
Microsoft Foundry AZURE_FOUNDRY_API_KEY azure_foundryAPIkey, azure_foundryAPIKey
TypeSafe TYPESAFE_API_KEY typesafeAPIkey, typesafeAPIKey

The 0.1.10 APIKey spelling takes precedence, followed by the older APIkey spelling and then the standard variable. Azure credentials must come from your Azure resource.

You can also inject an in-memory key using the new keyword-only argument LLM(..., api_key=secret_value). This takes precedence over all environment variables and works for existing providers too. Retrieve Key Vault secrets in your application or notebook and pass the value; this package has no dependency on a particular vault, Azure identity configuration or DataTeam filesystem path. Do not store keys in configuration files or commit them to source control.


Development

# Create a virtual environment
python -m venv .venv
source .venv/bin/activate  # macOS/Linux
.venv\Scripts\activate     # Windows

# Install dependencies
pip install -e .

Run the regression checks (simulated HTTP responses; no API credentials needed):

python -m unittest discover -s tests -v

License

MIT License © 2026 Your Name

Metadata

Release files for ehs-llm-client 0.2.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for ehs-llm-client 0.2.1
File Size Uploaded Provenance Actions
ehs_llm_client-0.2.1.tar.gz 35.3 kB Without provenance Details for ehs_llm_client-0.2.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for ehs-llm-client 0.2.1
File Interpreter ABI Platform Actions
ehs_llm_client-0.2.1-py3-none-any.whl Python 3 none any Details for ehs_llm_client-0.2.1-py3-none-any.whl

Total release size: 55.9 kB

Release files / ehs_llm_client-0.2.1.tar.gz

Download URL ehs_llm_client-0.2.1.tar.gz
Size 35.3 kB
Tags Source
SHA-256 checksum
How to use checksums
8aa8ede472d2a4b4bd94fd4c75eadbdb9c157fc30c5705b8b7a1567a61ae1dae
BLAKE2b-256 checksum
How to use checksums
605509efc5e51b627599fed50828061d357d8fbc57857f52a2265e0cd25124c5
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.13.5

Release files / ehs_llm_client-0.2.1-py3-none-any.whl

Download URL ehs_llm_client-0.2.1-py3-none-any.whl
Size 20.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
dbea9e90da0737f88a553bf01c29ed759947236664b282babf54cc1d8eaf197e
BLAKE2b-256 checksum
How to use checksums
4ddb823772a76c7d03576af0284ca629a115464411e227046ea69ce4a7d086c8
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.13.5

Security for ehs-llm-client 0.2.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page