kantan-llm 😺✨
A tiny Python library that removes the boring boilerplate (keys/URLs/provider selection) so you can call LLMs with a single get_llm() 💨
Big idea: set env vars for the providers/models you use, then just do get_llm("model-name") and it “just connects” 😺✨
Supported providers (roughly) 🌍
- OpenAI (Responses)
- Anthropic (Claude via OpenAI-compatible SDK)
- OpenRouter (OpenAI-compatible Chat)
- Google (Gemini via OpenAI-compatible Chat)
- LMStudio / Ollama / any OpenAI-compatible Chat
Install 📦
pip install kantan-llm
Quickstart 🚀
OpenAI (Responses API is the source of truth)
export OPENAI_API_KEY="sk-..."
from kantan_llm import get_llm
llm = get_llm("gpt-4.1-mini")
res = llm.responses.create(input="Say hi in one short line.")
print(res.output_text)
llm is OpenAI SDK compatible (unknown attributes delegate to the underlying client).
OpenAI-compatible (Chat Completions is the source of truth)
LMStudio (example: openai/gpt-oss-20b)
export LMSTUDIO_BASE_URL="http://192.168.11.16:1234" # `/v1` is optional
from kantan_llm import get_llm
llm = get_llm("openai/gpt-oss-20b", provider="lmstudio")
cc = llm.chat.completions.create(messages=[{"role": "user", "content": "Return exactly: OK"}], max_tokens=16)
print(cc.choices[0].message.content)
Ollama (example)
export OLLAMA_BASE_URL="http://localhost:11434" # `/v1` is optional
from kantan_llm import get_llm
llm = get_llm("llama3.2", provider="ollama")
cc = llm.chat.completions.create(messages=[{"role": "user", "content": "Return exactly: OK"}], max_tokens=16)
print(cc.choices[0].message.content)
Anthropic (Claude via OpenAI-compatible SDK)
export CLAUDE_API_KEY="sk-ant-..."
from kantan_llm import get_llm
llm = get_llm("claude-3-5-sonnet-latest") # if `CLAUDE_API_KEY` exists -> provider=anthropic (inferred)
cc = llm.chat.completions.create(messages=[{"role": "user", "content": "Return exactly: OK"}], max_tokens=16)
print(cc.choices[0].message.content)
OpenRouter (includes Claude, etc.)
export OPENROUTER_API_KEY="..."
from kantan_llm import get_llm
llm = get_llm("anthropic/claude-3.5-sonnet", provider="openrouter") # explicit is recommended (Anthropic takes precedence)
cc = llm.chat.completions.create(messages=[{"role": "user", "content": "Return exactly: OK"}], max_tokens=16)
print(cc.choices[0].message.content)
Google (Gemini via an OpenAI-compatible endpoint)
export GOOGLE_API_KEY="..."
from kantan_llm import get_llm
llm = get_llm("gemini-2.0-flash")
cc = llm.chat.completions.create(messages=[{"role": "user", "content": "Return exactly: OK"}], max_tokens=16)
print(cc.choices[0].message.content)
Provider rules 🧭
gpt-oss-*→ no fixed provider (uses env fallback; setprovider=if needed)gpt-*(exceptgpt-oss-*) →openaigemini-*→googleclaude-*→anthropic(ifCLAUDE_API_KEYis set) →openrouter(ifOPENROUTER_API_KEYis set) → otherwisecompat- If the model name is not recognizable, it picks the first available provider by env vars:
lmstudio→ollama→openrouter→anthropic→google
Explicit provider 🎯
from kantan_llm import get_llm
llm = get_llm("gpt-4.1-mini", provider="openai")
Fallback (order = priority) 🧯
from kantan_llm import get_llm
llm = get_llm("gpt-4.1-mini", providers=["openai", "lmstudio", "openrouter"])
Tracing / Tracer 🧵
By default, get_llm() enables a simple tracer that prints input/output (colorized) for each LLM call.
from kantan_llm import get_llm
from kantan_llm.tracing import trace
llm = get_llm("gpt-4.1-mini")
with trace("workflow"):
llm.responses.create(input="Say hi.")
More: docs/tracing.md
Async (ASGI) support
ASGI(FastAPI/Starlette)で event loop をブロックしないため、async 導線を提供します。
get_async_llm()(推奨)
- kantan-llm の保証(正規化/フォールバック/ガード/トレース)を async でも維持します。
Async streaming (KantanAsyncLLM)
KantanAsyncLLM では streaming API を提供し、最終応答でまとめてトレースします。
from kantan_llm import get_async_llm
llm = get_async_llm("gpt-4.1-mini")
async with llm.responses.stream(input="Say hi.") as stream:
async for _ in stream:
pass
final = await stream.get_final_response()
print(final.output_text)
Note: Some models (e.g. gpt-5-mini) may emit only response.output_item.* events without output_text/text deltas.
KantanAsyncLLM tries output_text first, then stream deltas, then output_item text; if none exists, the stream completes but the traced output can be empty.
get_async_llm_client()(Escape hatch)
AsyncOpenAIの raw client を返します(互換性最大化、Agents SDK 注入向け)。- 注意: raw client 返却では API ガード / 自動トレーシングは行いません。
- 代わりに
model/provider/base_urlを含む bundle を返し、正規化済み model 名を下流へ渡せます。
OpenAI Agents SDK integration
Agents SDK は AsyncOpenAI client を差し替え可能です。
- デフォルト client を差し替える:
set_default_openai_client(AsyncOpenAI(...))
- モデル単位で client を渡す:
OpenAIResponsesModel(..., openai_client=AsyncOpenAI(...))
In kantan-agents
kantan-agents (Agents SDK wrapper) uses the same two entry points:
set_default_openai_client(...)OpenAIResponsesModel(..., openai_client=...)
kantan-llm で Agents SDK を使う場合の推奨:
- 互換性優先:
bundle = get_async_llm_client(...)bundle.clientを Agents SDK に渡すbundle.model(正規化済み)を Agent/Model 側へ渡す
- kantan のガード/トレースも使いたい:
llm = get_async_llm(...)- ただし Agents SDK 側と二重トレースになり得るため、どちらでトレースするか方針を決める(下記)。
Tracing(二重計測を避ける)
Agents SDK 側にはトレーシング無効化の導線があります(例: set_tracing_disabled(True) や環境変数)。
運用では以下のどちらかを選びます。
- A) Agents SDK のトレースを有効、kantan 側トレースは無効(または raw client を使う)
- B) kantan のトレースを有効、Agents SDK 側トレースは無効
Search (SQLite) 🔎
Use SQLiteTracer as a lightweight search backend for traces/spans.
from kantan_llm.tracing import SpanQuery, TraceQuery
from kantan_llm.tracing.processors import SQLiteTracer
tracer = SQLiteTracer("traces.sqlite3")
traces = tracer.search_traces(query=TraceQuery(keywords=["hello"], limit=10))
spans = tracer.search_spans(query=SpanQuery(keywords=["hello"], limit=10))
More: docs/search.md
Tutorial: docs/tutorial_trace_analysis.md
Examples 📚
examples/tracing_basic.pyexamples/search_sqlite.py
Environment variables 🔐
- OpenAI
OPENAI_API_KEY(required)OPENAI_BASE_URL(optional)
- Generic compatible (
compat)KANTAN_LLM_BASE_URL(required)KANTAN_LLM_API_KEY(optional; falls back to a dummy value)
- LMStudio
LMSTUDIO_BASE_URL(required)
- Ollama
OLLAMA_BASE_URL(required)
- OpenRouter
OPENROUTER_API_KEY(required)
- Anthropic
CLAUDE_API_KEY(required)CLAUDE_BASE_URL(optional)
- Google
GOOGLE_API_KEY(required)GOOGLE_BASE_URL(optional)
Error example 💥
- Missing OpenAI key:
python -c 'from kantan_llm import get_llm; get_llm(\"gpt-4.1-mini\")'→[kantan-llm][E2] Missing OPENAI_API_KEY for provider: openai
Tests 🧪
Live integration tests (real APIs) are opt-in:
KANTAN_LLM_RUN_LIVE_TESTS=1 pytest -q -m integration
Metadata
Release files for kantan-llm 0.1.9
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| kantan_llm-0.1.9.tar.gz | 36.5 kB | Without provenance | Details for kantan_llm-0.1.9.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| kantan_llm-0.1.9-py3-none-any.whl | Python 3 | none | any | Details for kantan_llm-0.1.9-py3-none-any.whl |
Total release size: 71.4 kB
Release files / kantan_llm-0.1.9.tar.gz
| Download URL | kantan_llm-0.1.9.tar.gz |
|---|---|
| Size | 36.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
91320b798cc9f26d281b0d08d6c5a97ee7f2c5a7cc847393164133bda05c6300
|
|
BLAKE2b-256 checksum How to use checksums |
070939f13b51d7b5e8f945a869d35b66d3d533c945852fdcccf4ffc152661a9e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.10
|
Release files / kantan_llm-0.1.9-py3-none-any.whl
| Download URL | kantan_llm-0.1.9-py3-none-any.whl |
|---|---|
| Size | 34.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
c9c6ef256c69f0143d7aefec7924bf473543e60a6b5de14dc2a5491e6d88f10a
|
|
BLAKE2b-256 checksum How to use checksums |
f007a92752c59b76aa5ba3ca025b2745c5fa5b62ed126559a7d2aed88c50998b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.10
|
Security for kantan-llm 0.1.9
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.