ooai-llm 0.3.1

Typed LLM settings, LangChain-first factories, LiteLLM metadata enrichment, and callback helpers.

ooai-llm

Typed LLM settings, provider-aware model-string parsing, LangChain-first chat model creation, live provider model discovery, LiteLLM pricing enrichment, and usage/cost callback helpers for Python applications.

What This Is

ooai-llm is a small integration layer for application code that already wants to use LangChain model classes directly, but does not want to repeat the same provider configuration, model defaults, env-var handling, cache setup, metadata lookup, and usage accounting in every project.

It is not a router, proxy, agent framework, or hosted model catalog.

Features

• Typed ModelString parsing for bare, LangChain-style, and LiteLLM-style model names.
• Provider inference for OpenAI, Anthropic, Google GenAI, xAI, DeepSeek, and Mistral.
• AppSettings with provider credentials, default aliases, provider presets, cache settings, catalog settings, and LiteLLM settings.
• Native and app-prefixed credential env vars, such as OPENAI_API_KEY and OOAI_OPENAI_API_KEY.
• LangChain global cache bootstrap for SQLite, memory, SQLAlchemy, Redis, and Upstash Redis.
• create_llm(...), a thin wrapper around LangChain init_chat_model(...).
• create_llm_bundle(...), which returns the model, resolved metadata, and reasoning resolution together.
• Live model listing through provider SDKs or REST fallbacks.
• Provider-generic default refresh from live model catalogs or LiteLLM metadata.
• Opt-in automatic factory refresh so aliases such as latest can update at runtime.
• LangChain profile + LiteLLM pricing metadata in one ModelInfo object.
• Provider-aware reasoning kwargs for OpenAI, Anthropic, Gemini, xAI, DeepSeek, and Mistral.
• Usage and cost helpers for LangChain metadata and LiteLLM callbacks.
• Unit, integration, e2e, live-provider tests, coverage reports, docs builds, package builds, and PyPI release workflow.

Installation

Base package:

pip install ooai-llm

With PDM:

pdm add ooai-llm

Install only the provider extras you use:

pdm add ooai-llm[openai]
pdm add ooai-llm[anthropic]
pdm add ooai-llm[deepseek]
pdm add ooai-llm[mistral]
pdm add ooai-llm[litellm]
pdm add ooai-llm[redis]
pdm add ooai-llm[upstash]
pdm add ooai-llm[caches]

Gemini and xAI are available as ooai-llm[google] and ooai-llm[xai], but you can skip those extras entirely if you do not have those keys.

Factory Quick Start

from ooai_llm import AppSettings, configure_global_llm_cache, create_llm

settings = AppSettings()
configure_global_llm_cache(settings)

llm = create_llm(
    alias="testing",
    settings=settings,
    temperature=0,
    reasoning="fast",
)

print(type(llm).__name__)

Most applications only need the factory plus settings:

from ooai_llm import AppSettings, create_llm

settings = AppSettings()

testing_llm = create_llm(alias="testing", settings=settings, temperature=0)
reasoning_llm = create_llm(provider="anthropic", preset="reasoning", settings=settings)
explicit_llm = create_llm("openai:gpt-5.4-mini", settings=settings)

Use create_llm_bundle(...) when you want the created model and resolved metadata together:

from ooai_llm import create_llm_bundle

bundle = create_llm_bundle(
    alias="testing",
    reasoning="fast",
    temperature=0,
)

print(bundle.model.as_langchain())
print(bundle.metadata.identity.litellm_model)
print(bundle.reasoning.constructor_kwargs if bundle.reasoning else None)

Environment

The package accepts both native provider variables and OOAI_ aliases:

export OPENAI_API_KEY="..."
export ANTHROPIC_API_KEY="..."
export DEEPSEEK_API_KEY="..."
export MISTRAL_API_KEY="..."

export OOAI_OPENAI_API_KEY="..."
export OOAI_ANTHROPIC_API_KEY="..."
export OOAI_DEEPSEEK_API_KEY="..."
export OOAI_MISTRAL_API_KEY="..."

Google/Gemini and xAI variables are supported too, but are optional:

export GOOGLE_API_KEY="..."
export GEMINI_API_KEY="..."
export XAI_API_KEY="..."

Model Strings

from ooai_llm import ModelString

model = ModelString.parse("gpt-5.4-mini")
print(model.provider)       # Provider.OPENAI
print(model.model_name)     # gpt-5.4-mini
print(model.canonical())    # openai:gpt-5.4-mini
print(model.as_litellm())   # openai/gpt-5.4-mini

Settings And Defaults

from ooai_llm import AppSettings

settings = AppSettings()

print(settings.resolve_model(alias="cheap"))
print(settings.resolve_model(provider="anthropic", preset="reasoning"))
print(settings.default_llm_cache_path)

Default aliases and provider presets include:

• default
• latest
• cheap
• testing
• fast
• balanced
• reasoning
• coding
• vision

Live Model Discovery

from ooai_llm import AppSettings, ListModelsConfig, list_available_models

settings = AppSettings()
result = list_available_models(
    "openai",
    settings=settings,
    config=ListModelsConfig(limit=5),
)

for model in result.models:
    print(model.model_string, model.display_name)

Provider SDKs are preferred when installed. Supported REST fallbacks are used when SDK listing is unavailable or when you pass ListModelsConfig(prefer_sdk=False).

Refresh convenience factory defaults once at startup when you want aliases and provider presets to follow newer models:

from ooai_llm import AppSettings, refresh_model_defaults

settings = AppSettings()
refresh = refresh_model_defaults(
    settings,
    providers=["openai", "anthropic", "mistral"],
    source="auto",
)

settings = refresh.settings
print(settings.resolve_model(alias="latest"))
print(settings.resolve_model(provider="anthropic", preset="reasoning"))

Use update_model_defaults(...) when you want the refreshed settings plus a reusable override payload:

from ooai_llm import AppSettings, create_llm, update_model_defaults

update = update_model_defaults(
    AppSettings(),
    providers=["openai", "anthropic", "mistral"],
    source="litellm",
    output_format="env",
)

settings = update.settings
llm = create_llm(alias="latest", settings=settings)
print(update.output_text)

Or use the CLI to print or write those overrides:

ooai-llm models update --source litellm --providers openai,anthropic,mistral --format json
ooai-llm models update --source auto --provider openai --format env --output .env.models

Use source="litellm" to derive defaults from LiteLLM's local model registry without provider-listing credentials. Use source="provider" for live provider catalogs only.

Automatic refresh is opt-in for factory calls. Use this when you want create_llm(...) to refresh convenience defaults before resolving aliases:

from ooai_llm import AppSettings, create_llm

settings = AppSettings(
    llm={
        "auto_refresh_models": {
            "enabled": True,
            "source": "litellm",
            "providers": ["openai", "anthropic", "mistral"],
        }
    }
)

llm = create_llm(alias="latest", settings=settings)

The same setting can be enabled from .env:

OOAI_LLM__AUTO_REFRESH_MODELS__ENABLED=true
OOAI_LLM__AUTO_REFRESH_MODELS__SOURCE=litellm
OOAI_LLM__AUTO_REFRESH_MODELS__PROVIDERS='["openai","anthropic","mistral"]'

Automatic refresh uses a one-hour process-local cache by default. Set OOAI_LLM__AUTO_REFRESH_MODELS__CACHE_SECONDS=0 to refresh on every factory call, or pass force_model_refresh=True for one call.

Reasoning

from ooai_llm import ReasoningConfig, build_reasoning_resolution, create_llm

resolution = build_reasoning_resolution(
    model="openai:gpt-5.4-mini",
    reasoning="deep",
)
print(resolution.constructor_kwargs)

llm = create_llm(
    "anthropic:claude-sonnet-4-20250514",
    reasoning=ReasoningConfig(effort="medium", summary="auto"),
)

Metadata And Usage

from ooai_llm import BudgetPolicy, UsageRecorder, create_llm_bundle, make_litellm_cost_callback

bundle = create_llm_bundle(
    "openai:gpt-5.4-mini",
    reasoning="fast",
)

print(bundle.metadata.identity.litellm_model)
print(bundle.metadata.capabilities.raw_profile)
print(bundle.metadata.pricing.input_cost_per_token)

recorder = UsageRecorder()
callback = make_litellm_cost_callback(
    recorder,
    budget=BudgetPolicy(warn_total_tokens=5000),
)

Cache Bootstrap

from ooai_llm import AppSettings, configure_global_llm_cache

settings = AppSettings()
cache = configure_global_llm_cache(settings)
print(cache)

By default the SQLite cache is placed under:

{app_root}/.ooai/cache/llm/langchain_llm_cache.sqlite3

Override it with OOAI_LLM__CACHE__PATH or AppSettings(llm={"cache": {"path": ...}}).

Use Redis or Upstash Redis for a shared application cache:

settings = AppSettings(
    llm={
        "cache": {
            "backend": "redis",
            "redis_url": "redis://localhost:6379/0",
            "ttl": 3600,
        }
    }
)
configure_global_llm_cache(settings)

settings = AppSettings(
    llm={
        "cache": {
            "backend": "upstash_redis",
            "upstash_url": "...",
            "upstash_token": "...",
            "ttl": 3600,
        }
    }
)
configure_global_llm_cache(settings)

Development

Install the development dependencies:

pdm install -G test -G docs -G dev

Run the full checked suite with coverage:

pdm run pytest

Live provider tests are skipped by the default suite so local keys in .env do not trigger network calls accidentally.

Run tiers directly without the global coverage gate:

pdm run pytest -m unit --no-cov
pdm run pytest -m integration --no-cov
pdm run pytest -m "e2e and not live" --no-cov

Run live provider tests for your configured providers. This skips Gemini and xAI:

OOAI_LIVE_PROVIDERS=openai,anthropic,deepseek,mistral pdm run pytest -m live --no-cov

To make live e2e fail instead of skip when a selected provider is missing a key or SDK package:

OOAI_REQUIRE_LIVE=true OOAI_LIVE_PROVIDERS=openai,anthropic,deepseek,mistral pdm run pytest -m live --no-cov

AppSettings loads a local .env file. Keep real keys in .env, not .env.example.

Build docs and distributions:

pdm run sphinx-build -E -W --keep-going -b html docs docs/_build/html
pdm build
pdm run twine check dist/*

Project Layout

src/ooai_llm/
  cache.py       LangChain cache setup
  callbacks.py   usage and cost events
  catalog.py     live provider model listing
  factory.py     LangChain chat-model creation
  messages.py    message normalization
  metadata.py    LangChain + LiteLLM metadata
  providers.py   provider normalization
  reasoning.py   provider reasoning kwargs
  settings.py    Pydantic settings
  types.py       ModelString

docs/             Sphinx + MyST docs
examples/         runnable examples
tests/            unit, integration, e2e, and live tests

Publishing

The repository includes:

• .github/workflows/ci.yml for tests, coverage, docs build, and package build
• .github/workflows/docs.yml for standalone docs validation
• .github/workflows/release.yml for tagged PyPI releases with trusted publishing
• .readthedocs.yaml for Read the Docs builds

Before publishing, configure the PyPI trusted publisher for release.yml and environment pypi, import the repo in Read the Docs, and update docs/changelog.md.

License

MIT