llmshim

One interface, every LLM provider. The proxy server starts automatically — no setup needed.

Install

pip install llmshim

Configure

import llmshim

# Set API keys (writes to ~/.llmshim/config.toml — only needed once)
llmshim.configure(
    anthropic="sk-ant-...",
    openai="sk-...",
    gemini="AIza...",
    xai="xai-...",
    openrouter="sk-or-...",
)

Or from the CLI: llmshim configure

Self-hosted servers (vLLM / SGLang)

vLLM and SGLang are configured via environment variables (not config.toml) — the auto-spawned proxy inherits them from your Python process. Set them before your first call:

import os

os.environ["VLLM_BASE_URL"] = "http://localhost:8000/v1"
os.environ["VLLM_API_KEY"] = "..."       # optional
os.environ["SGLANG_BASE_URL"] = "http://localhost:30000/v1"
os.environ["SGLANG_API_KEY"] = "..."     # optional

Then address them via the model string — vllm/<served-model> or sglang/<served-model> (see the model table below).

Chat

import llmshim

resp = llmshim.chat("claude-sonnet-4-6", "What is Rust?")
print(resp["message"]["content"])

With options (all map to the API's provider-agnostic config):

resp = llmshim.chat(
    "openai/gpt-5.5",
    "Explain quicksort",
    max_tokens=500,
    temperature=0.7,
    top_p=0.9,
    top_k=40,
    stop=["\n\n"],
    reasoning_effort="high",
)

With message history:

resp = llmshim.chat("claude-sonnet-4-6", [
    {"role": "system", "content": "You are a pirate."},
    {"role": "user", "content": "Hello!"},
], max_tokens=500)

Streaming

for event in llmshim.stream("claude-sonnet-4-6", "Write a poem"):
    if event["type"] == "content":
        print(event["text"], end="", flush=True)
    elif event["type"] == "reasoning":
        pass  # thinking tokens
    elif event["type"] == "usage":
        print(f"\n[↑{event['input_tokens']} ↓{event['output_tokens']}]")

Multi-Model Conversations

Switch models mid-conversation. History carries over.

messages = [{"role": "user", "content": "What is a closure?"}]

r1 = llmshim.chat("claude-sonnet-4-6", messages, max_tokens=500)
print(f"Claude: {r1['message']['content']}")

messages.append({"role": "assistant", "content": r1["message"]["content"]})
messages.append({"role": "user", "content": "Now explain differently."})

r2 = llmshim.chat("gpt-5.5", messages, max_tokens=500)
print(f"GPT: {r2['message']['content']}")

Reasoning / Thinking

Two provider-agnostic knobs control reasoning; both are clamped to the nearest tier the target model supports:

reasoning_effort — "none", "low", "medium", "high", "xhigh", or "max"
reasoning_mode — "standard" (default) or "pro" (requests substantially more model work; native on OpenAI gpt-5.6/-pro, emulated as an effort bump elsewhere)

resp = llmshim.chat(
    "claude-sonnet-5",
    "Solve: x^2 - 5x + 6 = 0",
    max_tokens=4000,
    reasoning_effort="high",
    reasoning_mode="pro",
)
print(resp["reasoning"])        # thinking content
print(resp["message"]["content"])  # answer

For full native control, bypass the unified mapping with a namespaced provider_config (see below), e.g. provider_config={"x-anthropic": {"thinking": {"type": "enabled", "budget_tokens": 4000}}}.

Provider-Specific Controls (`provider_config`)

provider_config merges into the request root and carries anything the unified config doesn't cover. Native provider controls MUST be namespaced per provider (x-anthropic, x-openai, x-gemini, x-openrouter, x-vllm, x-sglang); it also carries the top-level tools, response_format, and reasoning_summary keys.

resp = llmshim.chat(
    "anthropic/claude-sonnet-5",
    "Solve this step by step: 17 * 23",
    max_tokens=4000,
    provider_config={
        # native Anthropic extended-thinking control
        "x-anthropic": {"thinking": {"type": "enabled", "budget_tokens": 4000}},
        # structured output
        "response_format": {"type": "json_object"},
    },
)

OpenRouter routing preferences use the x-openrouter namespace:

resp = llmshim.chat(
    "openrouter/anthropic/claude-sonnet-4.5",
    "Hello",
    max_tokens=200,
    provider_config={"x-openrouter": {"provider": {"sort": "throughput"}}},
)

Tool Use / Function Calling

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]

resp = llmshim.chat("claude-sonnet-4-6", "Weather in Tokyo?", max_tokens=500, tools=tools)
for tc in resp["message"].get("tool_calls", []):
    print(f"{tc['function']['name']}({tc['function']['arguments']})")

Tools are accepted in OpenAI Chat Completions format and auto-translated to each provider's native format.

Fallback Chains

resp = llmshim.chat(
    "anthropic/claude-sonnet-4-6",
    "Hello",
    max_tokens=100,
    fallback=["openai/gpt-5.6-sol", "gemini/gemini-3.5-flash"],
)

Error Handling

Non-streaming errors (bad model, unknown provider, provider failures) raise LlmShimError, which carries the API's structured error fields:

try:
    llmshim.chat("unknown/model", "hi")
except llmshim.LlmShimError as e:
    print(e.status_code)  # 400
    print(e.code)         # "bad_request"
    print(e.message)      # human-readable message

Streaming errors that occur mid-stream instead arrive as an error event (event["type"] == "error"); an HTTP error before the stream starts still raises LlmShimError.

Types

Spec-faithful TypedDict definitions are available in llmshim.types (and the common ones are re-exported at the top level) for static type-checking:

from llmshim.types import ChatResponse, StreamEvent, Message, Config

resp: ChatResponse = llmshim.chat("claude-sonnet-4-6", "hi")

Available: ChatRequest, ChatResponse, Config, Message, ToolCall, Usage, ResponseMessage, ModelEntry, ModelsResponse, HealthResponse, ErrorResponse, and the StreamEvent union (ContentEvent, ReasoningEvent, ToolCallEvent, UsageEvent, DoneEvent, ErrorEvent).

Other

llmshim.models()   # list available models
llmshim.health()   # {"status": "ok", "providers": [...]}

How It Works

On first call, the package:

Finds the llmshim binary (bundled, on PATH, or in repo)
Starts the proxy on a random localhost port
Routes your request through it
Server stops automatically when Python exits

No Docker, no background services, no manual server management.

Development

The unit tests under tests/ are fully mocked — they run a local HTTP server returning canned JSON and SSE, so they need no API keys and make no real provider calls:

pip install httpx pytest
pytest tests/

test_e2e.py is a separate LIVE suite that spawns the real binary and makes billed provider calls; run it only when you deliberately want to hit real APIs.

Supported Models

Provider	Models
OpenAI	`gpt-5.6-sol`, `gpt-5.6-terra`, `gpt-5.6-luna`, `gpt-5.5`, `gpt-5.5-pro`, `gpt-5.4`, `gpt-5.4-pro`, `gpt-5.4-mini`, `gpt-5.4-nano`
Anthropic	`claude-opus-5`, `claude-opus-4-8`, `claude-sonnet-5`, `claude-opus-4-7`, `claude-opus-4-6`, `claude-sonnet-4-6`, `claude-haiku-4-5-20251001`
Gemini	`gemini-3.7-flash`, `gemini-3.6-flash`, `gemini-3.5-flash`, `gemini-3.5-flash-lite`, `gemini-3.1-flash-lite`
xAI	`grok-4.6`, `grok-4.5`, `grok-4.3`, `grok-4.20-multi-agent-beta-0309`, `grok-4.20-beta-0309-reasoning`, `grok-4.20-beta-0309-non-reasoning`

Call llmshim.models() for the live list filtered to your configured providers.

OpenRouter & self-hosted (vLLM / SGLang)

These providers are addressed by the model string plus environment variables — any model the upstream serves is reachable, so they aren't in the table above.

Provider	Address as	Env vars	Native controls
OpenRouter	`openrouter/<vendor>/<model>` (e.g. `openrouter/anthropic/claude-sonnet-4.5`)	`OPENROUTER_API_KEY` (or `llmshim.configure(openrouter=...)`)	`provider_config={"x-openrouter": {...}}` (`provider`, `models`, `transforms`)
vLLM	`vllm/<served-model>`	`VLLM_BASE_URL` (+ optional `VLLM_API_KEY`)	`provider_config={"x-vllm": {...}}`
SGLang	`sglang/<served-model>`	`SGLANG_BASE_URL` (+ optional `SGLANG_API_KEY`)	`provider_config={"x-sglang": {...}}`

Development Status
- 4 - Beta
Intended Audience
- Developers
License
- OSI Approved :: MIT License
Programming Language
- Python :: 3
- Rust
Topic
- Software Development :: Libraries

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

llmshim-0.3.4.tar.gz (1.0 MB view details)

Uploaded Aug 20, 2026 Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

llmshim-0.3.4-py3-none-win_amd64.whl (3.2 MB view details)

Uploaded Aug 20, 2026 Python 3Windows x86-64

llmshim-0.3.4-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.3 MB view details)

Uploaded Aug 20, 2026 Python 3manylinux: glibc 2.17+ x86-64

llmshim-0.3.4-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.1 MB view details)

Uploaded Aug 20, 2026 Python 3manylinux: glibc 2.17+ ARM64

llmshim-0.3.4-py3-none-macosx_11_0_arm64.whl (3.1 MB view details)

Uploaded Aug 20, 2026 Python 3macOS 11.0+ ARM64

llmshim-0.3.4-py3-none-macosx_10_12_x86_64.whl (3.3 MB view details)

Uploaded Aug 20, 2026 Python 3macOS 10.12+ x86-64

File details

Details for the file llmshim-0.3.4.tar.gz.

File metadata

Download URL: llmshim-0.3.4.tar.gz
Upload date: Aug 20, 2026
Size: 1.0 MB
Tags: Source
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for llmshim-0.3.4.tar.gz
Algorithm	Hash digest
SHA256	`6d50de4026dbdb912520c5d663b48ec8819ed8b028a78f69e0e271e043862e7e`
MD5	`aaf3c1b64f23022da4eeca683b9033e0`
BLAKE2b-256	`412bff5376036a19beec6d00c07efdde373f7382024154d509343482ef42446c`

See more details on using hashes here.

Provenance

The following attestation bundles were made for llmshim-0.3.4.tar.gz:

Publisher: release.yml on sanjay920/llmshim

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: llmshim-0.3.4.tar.gz
- Subject digest: 6d50de4026dbdb912520c5d663b48ec8819ed8b028a78f69e0e271e043862e7e
- Sigstore transparency entry: 2526581786
- Sigstore integration time: Aug 20, 2026
Source repository:
- Permalink: sanjay920/llmshim@ceef8550f75dcddb61e7447570bcbc39a6544b9c
- Branch / Tag: refs/tags/v0.3.4
- Owner: https://github.com/sanjay920
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: release.yml@ceef8550f75dcddb61e7447570bcbc39a6544b9c
- Trigger Event: push

File details

Details for the file llmshim-0.3.4-py3-none-win_amd64.whl.

File metadata

Download URL: llmshim-0.3.4-py3-none-win_amd64.whl
Upload date: Aug 20, 2026
Size: 3.2 MB
Tags: Python 3, Windows x86-64
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for llmshim-0.3.4-py3-none-win_amd64.whl
Algorithm	Hash digest
SHA256	`20235b114ab3fe44ff466d14aad048dfafcf94c83a41368ae6e97f5669064808`
MD5	`2a92249db60d017e5fe7ff74f5297427`
BLAKE2b-256	`49af8a5971228a64ce7ec8df2bf89d29c3ff291064d69b50b227947279f3566e`

See more details on using hashes here.

Provenance

The following attestation bundles were made for llmshim-0.3.4-py3-none-win_amd64.whl:

Publisher: release.yml on sanjay920/llmshim

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: llmshim-0.3.4-py3-none-win_amd64.whl
- Subject digest: 20235b114ab3fe44ff466d14aad048dfafcf94c83a41368ae6e97f5669064808
- Sigstore transparency entry: 2526582034
- Sigstore integration time: Aug 20, 2026
Source repository:
- Permalink: sanjay920/llmshim@ceef8550f75dcddb61e7447570bcbc39a6544b9c
- Branch / Tag: refs/tags/v0.3.4
- Owner: https://github.com/sanjay920
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: release.yml@ceef8550f75dcddb61e7447570bcbc39a6544b9c
- Trigger Event: push

File details

Details for the file llmshim-0.3.4-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

Download URL: llmshim-0.3.4-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Upload date: Aug 20, 2026
Size: 3.3 MB
Tags: Python 3, manylinux: glibc 2.17+ x86-64
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for llmshim-0.3.4-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm	Hash digest
SHA256	`2870255d8b71810e07ef53fc05f2b95142b452b3a10834804eb4aa0d2f67060f`
MD5	`8828cbf627e7a5e4d92c69982a4cc6c7`
BLAKE2b-256	`5e140cbd3afe43a83c4d676ed5686b3171008ed90fe608bb40805670e26ad3ea`

See more details on using hashes here.

Provenance

The following attestation bundles were made for llmshim-0.3.4-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on sanjay920/llmshim

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: llmshim-0.3.4-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
- Subject digest: 2870255d8b71810e07ef53fc05f2b95142b452b3a10834804eb4aa0d2f67060f
- Sigstore transparency entry: 2526581972
- Sigstore integration time: Aug 20, 2026
Source repository:
- Permalink: sanjay920/llmshim@ceef8550f75dcddb61e7447570bcbc39a6544b9c
- Branch / Tag: refs/tags/v0.3.4
- Owner: https://github.com/sanjay920
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: release.yml@ceef8550f75dcddb61e7447570bcbc39a6544b9c
- Trigger Event: push

File details

Details for the file llmshim-0.3.4-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

Download URL: llmshim-0.3.4-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Upload date: Aug 20, 2026
Size: 3.1 MB
Tags: Python 3, manylinux: glibc 2.17+ ARM64
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for llmshim-0.3.4-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm	Hash digest
SHA256	`57e52ef62a714716da156a0ffbab8b26dd998db90b4a4734b8c39ccd2565374e`
MD5	`3ba6a7dca705dc66d0ba3d4390246089`
BLAKE2b-256	`b40486b6575fc2a1b1e5c28f601618ce850c67bdfbdc5e452ab3eaf863d8a1d7`

See more details on using hashes here.

Provenance

The following attestation bundles were made for llmshim-0.3.4-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: release.yml on sanjay920/llmshim

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: llmshim-0.3.4-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
- Subject digest: 57e52ef62a714716da156a0ffbab8b26dd998db90b4a4734b8c39ccd2565374e
- Sigstore transparency entry: 2526581925
- Sigstore integration time: Aug 20, 2026
Source repository:
- Permalink: sanjay920/llmshim@ceef8550f75dcddb61e7447570bcbc39a6544b9c
- Branch / Tag: refs/tags/v0.3.4
- Owner: https://github.com/sanjay920
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: release.yml@ceef8550f75dcddb61e7447570bcbc39a6544b9c
- Trigger Event: push

File details

Details for the file llmshim-0.3.4-py3-none-macosx_11_0_arm64.whl.

File metadata

Download URL: llmshim-0.3.4-py3-none-macosx_11_0_arm64.whl
Upload date: Aug 20, 2026
Size: 3.1 MB
Tags: Python 3, macOS 11.0+ ARM64
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for llmshim-0.3.4-py3-none-macosx_11_0_arm64.whl
Algorithm	Hash digest
SHA256	`5d2d0ddabc79f6b755536fde657d60e0d8d3319b002f352a55d49138677ccfdd`
MD5	`22a8071b34314189a7810e17a742c01b`
BLAKE2b-256	`d0126231faf25fde1a66a9d449bb46bc071cd6a1f92d4d8f548bf239eee79f51`

See more details on using hashes here.

Provenance

The following attestation bundles were made for llmshim-0.3.4-py3-none-macosx_11_0_arm64.whl:

Publisher: release.yml on sanjay920/llmshim

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: llmshim-0.3.4-py3-none-macosx_11_0_arm64.whl
- Subject digest: 5d2d0ddabc79f6b755536fde657d60e0d8d3319b002f352a55d49138677ccfdd
- Sigstore transparency entry: 2526581840
- Sigstore integration time: Aug 20, 2026
Source repository:
- Permalink: sanjay920/llmshim@ceef8550f75dcddb61e7447570bcbc39a6544b9c
- Branch / Tag: refs/tags/v0.3.4
- Owner: https://github.com/sanjay920
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: release.yml@ceef8550f75dcddb61e7447570bcbc39a6544b9c
- Trigger Event: push

File details

Details for the file llmshim-0.3.4-py3-none-macosx_10_12_x86_64.whl.

File metadata

Download URL: llmshim-0.3.4-py3-none-macosx_10_12_x86_64.whl
Upload date: Aug 20, 2026
Size: 3.3 MB
Tags: Python 3, macOS 10.12+ x86-64
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for llmshim-0.3.4-py3-none-macosx_10_12_x86_64.whl
Algorithm	Hash digest
SHA256	`c1a42523e55236ae3d9f88b470310f7f307a330435febd29672a1b70b2de435f`
MD5	`1a6966a760e2103f4b954e4428031597`
BLAKE2b-256	`bb5e30222cbe20039abe9033d5ea5b451fd9241d740eefb92d1d277d19968589`

See more details on using hashes here.

Provenance

The following attestation bundles were made for llmshim-0.3.4-py3-none-macosx_10_12_x86_64.whl:

Publisher: release.yml on sanjay920/llmshim

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: llmshim-0.3.4-py3-none-macosx_10_12_x86_64.whl
- Subject digest: c1a42523e55236ae3d9f88b470310f7f307a330435febd29672a1b70b2de435f
- Sigstore transparency entry: 2526581880
- Sigstore integration time: Aug 20, 2026
Source repository:
- Permalink: sanjay920/llmshim@ceef8550f75dcddb61e7447570bcbc39a6544b9c
- Branch / Tag: refs/tags/v0.3.4
- Owner: https://github.com/sanjay920
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: release.yml@ceef8550f75dcddb61e7447570bcbc39a6544b9c
- Trigger Event: push

Release history Release notifications | RSS feed

This release

0.3.4

Aug 20, 2026

0.3.3

Aug 20, 2026

0.3.2

Aug 14, 2026

0.3.1

Jul 26, 2026

0.3.0

Jul 26, 2026

0.2.3

Jul 25, 2026

0.2.2

Jul 18, 2026

0.2.1

Jul 17, 2026

0.2.0

Jul 16, 2026

0.1.26

Jul 13, 2026

0.1.25

Jul 13, 2026

0.1.24

Jul 8, 2026

0.1.23

Jul 4, 2026

0.1.22

Jul 3, 2026

0.1.21

Apr 30, 2026

0.1.20

Apr 25, 2026

0.1.18

Apr 4, 2026

0.1.17

Apr 4, 2026

0.1.16

Mar 19, 2026

0.1.15

Mar 19, 2026

0.1.14

Mar 19, 2026

0.1.13

Mar 18, 2026

0.1.12

Mar 17, 2026

0.1.11

Mar 17, 2026

0.1.10

Mar 14, 2026

0.1.9

Mar 13, 2026

0.1.8

Mar 12, 2026

0.1.7

Mar 12, 2026

0.1.6

Mar 12, 2026

0.1.4

Mar 12, 2026

0.1.3

Mar 12, 2026

0.1.2

Mar 12, 2026

0.1.0

Mar 11, 2026

llmshim 0.3.4

llmshim

Install

Configure

Self-hosted servers (vLLM / SGLang)

Chat

Streaming

Multi-Model Conversations

Reasoning / Thinking

Provider-Specific Controls (provider_config)

Tool Use / Function Calling

Fallback Chains

Error Handling

Types

Other

How It Works

Development

Supported Models

OpenRouter & self-hosted (vLLM / SGLang)

Project links

Key dates

1 maintainer

GitHub Statistics

License

Requires

Classifiers

Download files

Source Distribution

Built Distributions

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

Release history Release notifications | RSS feed

Provider-Specific Controls (`provider_config`)