llmshim
One interface, every LLM provider. The proxy server starts automatically — no setup needed.
Install
pip install llmshim
Configure
import llmshim
# Set API keys (writes to ~/.llmshim/config.toml — only needed once)
llmshim.configure(
anthropic="sk-ant-...",
openai="sk-...",
gemini="AIza...",
xai="xai-...",
openrouter="sk-or-...",
)
Or from the CLI: llmshim configure
Self-hosted servers (vLLM / SGLang)
vLLM and SGLang are configured via environment variables (not
config.toml) — the auto-spawned proxy inherits them from your Python
process. Set them before your first call:
import os
os.environ["VLLM_BASE_URL"] = "http://localhost:8000/v1"
os.environ["VLLM_API_KEY"] = "..." # optional
os.environ["SGLANG_BASE_URL"] = "http://localhost:30000/v1"
os.environ["SGLANG_API_KEY"] = "..." # optional
Then address them via the model string — vllm/<served-model> or
sglang/<served-model> (see the model table below).
Chat
import llmshim
resp = llmshim.chat("claude-sonnet-5", "What is Rust?")
print(resp["message"]["content"])
With options (all map to the API's provider-agnostic config):
resp = llmshim.chat(
"openai/gpt-5.6-sol",
"Explain quicksort",
max_tokens=500,
temperature=0.7,
top_p=0.9,
top_k=40,
stop=["\n\n"],
reasoning_effort="high",
)
With message history:
resp = llmshim.chat("claude-sonnet-5", [
{"role": "system", "content": "You are a pirate."},
{"role": "user", "content": "Hello!"},
], max_tokens=500)
Streaming
for event in llmshim.stream("claude-sonnet-5", "Write a poem"):
if event["type"] == "content":
print(event["text"], end="", flush=True)
elif event["type"] == "reasoning":
pass # thinking tokens
elif event["type"] == "usage":
print(f"\n[↑{event['input_tokens']} ↓{event['output_tokens']}]")
Multi-Model Conversations
Switch models mid-conversation. History carries over.
messages = [{"role": "user", "content": "What is a closure?"}]
r1 = llmshim.chat("claude-sonnet-5", messages, max_tokens=500)
print(f"Claude: {r1['message']['content']}")
messages.append({"role": "assistant", "content": r1["message"]["content"]})
messages.append({"role": "user", "content": "Now explain differently."})
r2 = llmshim.chat("gpt-5.6-sol", messages, max_tokens=500)
print(f"GPT: {r2['message']['content']}")
Reasoning / Thinking
Two provider-agnostic knobs control reasoning; both are clamped to the nearest tier the target model supports:
reasoning_effort—"none","low","medium","high","xhigh", or"max"reasoning_mode—"standard"(default) or"pro"(requests substantially more model work; native on OpenAI gpt-5.6/-pro, emulated as an effort bump elsewhere)
resp = llmshim.chat(
"claude-sonnet-5",
"Solve: x^2 - 5x + 6 = 0",
max_tokens=4000,
reasoning_effort="high",
reasoning_mode="pro",
)
print(resp["reasoning"]) # thinking content
print(resp["message"]["content"]) # answer
For full native control, bypass the unified mapping with a namespaced
provider_config (see below), e.g.
provider_config={"x-anthropic": {"thinking": {"type": "adaptive"}, "output_config": {"effort": "high"}}}.
Provider-Specific Controls (provider_config)
provider_config merges into the request root and carries anything the
unified config doesn't cover. Native provider controls MUST be namespaced
per provider (x-anthropic, x-openai, x-gemini, x-openrouter, x-vllm,
x-sglang); it also carries the top-level tools, response_format, and
reasoning_summary keys.
resp = llmshim.chat(
"anthropic/claude-sonnet-5",
"Solve this step by step: 17 * 23",
max_tokens=4000,
provider_config={
# native Anthropic adaptive-thinking control
"x-anthropic": {"thinking": {"type": "adaptive"}, "output_config": {"effort": "high"}},
},
)
OpenRouter routing preferences use the x-openrouter namespace:
resp = llmshim.chat(
"openrouter/anthropic/claude-sonnet-5",
"Hello",
max_tokens=200,
provider_config={"x-openrouter": {"provider": {"sort": "throughput"}}},
)
Tool Use / Function Calling
tools = [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"],
},
},
}]
resp = llmshim.chat("claude-sonnet-5", "Weather in Tokyo?", max_tokens=500, tools=tools)
for tc in resp["message"].get("tool_calls", []):
print(f"{tc['function']['name']}({tc['function']['arguments']})")
Tools are accepted in OpenAI Chat Completions format and auto-translated to each provider's native format.
Fallback Chains
resp = llmshim.chat(
"anthropic/claude-sonnet-5",
"Hello",
max_tokens=100,
fallback=["openai/gpt-5.6-sol", "gemini/gemini-3.8-flash"],
)
Error Handling
Non-streaming errors (bad model, unknown provider, provider failures) raise
LlmShimError, which carries the API's structured error fields:
try:
llmshim.chat("unknown/model", "hi")
except llmshim.LlmShimError as e:
print(e.status_code) # 400
print(e.code) # "bad_request"
print(e.message) # human-readable message
Streaming errors that occur mid-stream instead arrive as an error event
(event["type"] == "error"); an HTTP error before the stream starts still
raises LlmShimError.
Types
Spec-faithful TypedDict definitions are available in llmshim.types (and the
common ones are re-exported at the top level) for static type-checking:
from llmshim.types import ChatResponse, StreamEvent, Message, Config
resp: ChatResponse = llmshim.chat("claude-sonnet-5", "hi")
Available: ChatRequest, ChatResponse, Config, Message, ToolCall,
Usage, ResponseMessage, ModelEntry, ModelsResponse, HealthResponse,
ErrorResponse, and the StreamEvent union (ContentEvent, ReasoningEvent,
ToolCallEvent, UsageEvent, DoneEvent, ErrorEvent).
Other
llmshim.models() # list available models
llmshim.health() # {"status": "ok", "providers": [...]}
How It Works
On first call, the package:
- Finds the
llmshimbinary (bundled, on PATH, or in repo) - Starts the proxy on a random localhost port
- Routes your request through it
- Server stops automatically when Python exits
No Docker, no background services, no manual server management.
Development
The unit tests under tests/ are fully mocked — they run a local HTTP server
returning canned JSON and SSE, so they need no API keys and make no real
provider calls:
pip install httpx pytest
pytest tests/
test_e2e.py is a separate LIVE suite that spawns the real binary and makes
billed provider calls; run it only when you deliberately want to hit real APIs.
Advertised Models
| Provider | Models |
|---|---|
| OpenAI | gpt-6-astra, gpt-5.6-sol, gpt-5.6-terra, gpt-5.6-luna |
| Anthropic | claude-fable-5-1, claude-opus-5, claude-sonnet-5, claude-haiku-4-5-20251001 |
| Gemini | gemini-3.8-flash, gemini-3.5-flash-lite |
| xAI | grok-4.6 |
Call llmshim.models() for the live list filtered to your configured providers.
OpenRouter & self-hosted (vLLM / SGLang)
These providers are addressed by the model string plus environment variables — any model the upstream serves is reachable, so they aren't in the table above.
| Provider | Address as | Env vars | Native controls |
|---|---|---|---|
| OpenRouter | openrouter/<vendor>/<model> (e.g. openrouter/anthropic/claude-sonnet-5) |
OPENROUTER_API_KEY (or llmshim.configure(openrouter=...)) |
provider_config={"x-openrouter": {...}} (provider, models, transforms) |
| vLLM | vllm/<served-model> |
VLLM_BASE_URL (+ optional VLLM_API_KEY) |
provider_config={"x-vllm": {...}} |
| SGLang | sglang/<served-model> |
SGLANG_BASE_URL (+ optional SGLANG_API_KEY) |
provider_config={"x-sglang": {...}} |
1 maintainer
Metadata
1 maintainer
Release files for llmshim 0.8.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| llmshim-0.8.1.tar.gz | 1.7 MB | Details |
Built distributions (wheels)
| File | Reset | |||
|---|---|---|---|---|
| llmshim-0.8.1-py3-none-win_amd64.whl | Python 3 | none | Windows x86-64 | Details |
| llmshim-0.8.1-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl | Python 3 | none | Linux glibc 2.17+ x86-64 | Details |
| llmshim-0.8.1-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl | Python 3 | none | Linux glibc 2.17+ ARM64 | Details |
| llmshim-0.8.1-py3-none-macosx_11_0_arm64.whl | Python 3 | none | macOS 11.0+ ARM64 | Details |
| llmshim-0.8.1-py3-none-macosx_10_12_x86_64.whl | Python 3 | none | macOS 10.12+ x86-64 | Details |
Total release size: 32.0 MB
Release files / llmshim-0.8.1.tar.gz
| Download URL | llmshim-0.8.1.tar.gz |
|---|---|
| Size | 1.7 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
e108b4a91b7794e534a50e15d21ed3ec1289a33735ad2d6f6026d24d1a524034
|
|
BLAKE2b-256 checksum How to use checksums |
40757c434548fe8271aabc931211240ca915ed7eeefb6eaa08c409afc52beedb
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 22, 2026.
Transparency logRelease files / llmshim-0.8.1-py3-none-win_amd64.whl
| Download URL | llmshim-0.8.1-py3-none-win_amd64.whl |
|---|---|
| Size | 6.2 MB |
| Tags | Python 3 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
c4baee497dbe0a44b6d51216d2ca84b1e072394b562134c882fc0e65de8d4ab2
|
|
BLAKE2b-256 checksum How to use checksums |
b65b11bf003320fd6f9d357467fade09fabc0fa0743319119b5cf558d3914c05
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 22, 2026.
Transparency logRelease files / llmshim-0.8.1-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | llmshim-0.8.1-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 6.3 MB |
| Tags | Linux glibc 2.17+ x86-64 Python 3 |
|
SHA-256 checksum How to use checksums |
ecf05a418fb14f4f503c2531e0d47f2ab1aaa1ca69878f940916a3ee533af695
|
|
BLAKE2b-256 checksum How to use checksums |
f6dce0979a9997e28dc500044f456775509beab035dd3a51d019116d0802420d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 22, 2026.
Transparency logRelease files / llmshim-0.8.1-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
| Download URL | llmshim-0.8.1-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl |
|---|---|
| Size | 5.9 MB |
| Tags | Linux glibc 2.17+ ARM64 Python 3 |
|
SHA-256 checksum How to use checksums |
997315c13562ed4c696eee42f05fedeb5910e8c56f772aa10ce1d9990b86e7c9
|
|
BLAKE2b-256 checksum How to use checksums |
c2b9ff006e38ff26facd353db480e97244e987012e08cab767950004b0735889
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 22, 2026.
Transparency logRelease files / llmshim-0.8.1-py3-none-macosx_11_0_arm64.whl
| Download URL | llmshim-0.8.1-py3-none-macosx_11_0_arm64.whl |
|---|---|
| Size | 5.8 MB |
| Tags | Python 3 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
bd3dd96980084d994d4fdf2eef7160039cdd34048e0eaa893dcf130eb389046e
|
|
BLAKE2b-256 checksum How to use checksums |
290fcc6642983b5128d4a3f84846e6c53ece2c4f53bc2a49348b63d8a95554f5
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 22, 2026.
Transparency logRelease files / llmshim-0.8.1-py3-none-macosx_10_12_x86_64.whl
| Download URL | llmshim-0.8.1-py3-none-macosx_10_12_x86_64.whl |
|---|---|
| Size | 6.1 MB |
| Tags | Python 3 macOS 10.12+ x86-64 |
|
SHA-256 checksum How to use checksums |
86e5a95b519d6a8e0701517626494cb5de892dde0c8bf78b427f7451cc54c82c
|
|
BLAKE2b-256 checksum How to use checksums |
c0da7d22dbed04561b1b4e21cba47d42c0cca4539684f6a5e1875a4828f63b16
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 22, 2026.
Transparency log
