justllm 0.4.0

Production LLM calls. Just the three lines. Reliability, native caching, and reversible context compression on by default.

justllm

Production LLM calls. Just the three lines.

from justllm import LLM

llm = LLM("anthropic/claude-opus-4-8")
llm("Summarize this contract.")

That one call already does what you'd normally wire by hand — on by default, zero config:

• Context compression — Headroom shrinks tool output 50–95% before it reaches the model
• Prompt-cache optimization — provider-optimal cache breakpoints (Anthropic / OpenAI / Google)
• Reliability — retry with backoff, then fail over across providers

pip install 'justllm[all]'

More, when you want it

Same three-line surface — each of these is one call or one kwarg:

llm.extract(Invoice, text)                    # structured output (validated Pydantic)
llm.stream("...")                             # token streaming
await llm.acall("...")                        # async
llm.agent(system="...").run("...")            # tool-calling loop
LLM(router=Cascade(small=cheap, large=big))   # cheap first, escalate when needed

Plus OpenTelemetry tracing with the per-call cost the spec omits ([otel]), Langfuse-backed prompts, semantic cascade escalation, exact-match caching — all opt-in. The point: every one of these is SOTA under the hood and a one-liner on top.

Runnable recipes for all of it: cookbook →

Why

The ecosystem is split: powerful but heavy (LiteLLM, LangChain), or simple but thin (aisuite, any-llm). justllm is the middle — every optimization on, behind a three-line surface. The discipline is the product.

---

Alpha. Wiring is tested on CI (Python 3.10–3.13); call paths are validated live.

Cookbook · Roadmap · Changelog · Contributing · MIT