llm-catalog-litellm

LiteLLM CustomLLM plugin for llm-catalog. One implementation serves both in-process use and the central proxy.

For in-process use, call register() once to wire the handler into LiteLLM.

from llm_catalog.litellm import register

register()  # reads ai-sdk-catalog.json (LLM_CATALOG_CONFIG or default path)

import litellm

resp = litellm.completion(
    model="examplegw/fast",
    messages=[{"role": "user", "content": "hi"}],
)

For the proxy, reference llm_catalog.litellm.handler from config.yaml's custom_provider_map (no register() call needed).

The handler resolves each model from ai-sdk-catalog.json itself (the JSON config shared verbatim with — and named after — ai-sdk-catalog; the legacy llm-catalog.json remains a deprecated fallback), so the proxy config never needs gateway details in litellm_params, sidestepping LiteLLM issue #18216.

Requires Python 3.12–3.14: LiteLLM declares requires-python <3.15, and this adapter mirrors that cap until LiteLLM lifts it. LiteLLM is built on httpx and requires openai<3, while llm-catalog-pydantic-ai and llm-catalog-ai-sdk need openai>=3 (the httpx2 line), so install this adapter in its own environment rather than alongside those two.

See the repository README for the proxy operations guide and the verification notes (§9), including whether LiteLLM honours a custom httpx client (the route-1/route-2 decision in §6.2).

import namespace: llm_catalog.litellm