llmbroker
Use multiple free LLM providers through one reliable API. llmbroker selects an available model, recovers from provider errors and rate limits, and learns which models work best for each task.
Start with no configuration file, paid subscription, or heavy orchestration framework. Add production storage, per-user keys, or a specific paid model only when you need them.
pip install llmbroker
llmbroker env freetier > .env # required key names and links to obtain them
import llmbroker
broker = llmbroker.Broker()
reply = broker.ask("Explain decorators in one sentence")
print(reply.text)
Add any provider keys you have; models without keys are skipped automatically.
| Resilient by default | Automatically tries another model after rate limits and provider failures |
| Recovery without delays | Rechecks a failed model alongside a working one; disable with parallel_recovery=False |
| Lower latency on demand | fastest_of=2 queries multiple models and returns the first complete reply |
| Improves with feedback | record_quality() adapts model selection independently for each operation |
| Complete application API | Sync and async calls, streaming, chat, tools, and built-in tool loops |
| Direct premium access | Broker(direct=["opus"]) calls a specific paid model through a stable alias |
| Ready to scale | SQLite, PostgreSQL, MongoDB, shared journals, per-user keys, and pluggable secrets |
| Observable | Pool state, call history, statistics, trace_id, and availability alerts |
Development
Do not forget to run . ./activate.sh. It needs uv installed.
Use pre-commit hooks for code quality:
pre-commit install
Install invoke preferably with uv tool:
uv tool install invoke
For a list of available scripts run invoke --list; for details on one, invoke <script> --help.
The bundled freetier preset drifts as providers change their free tiers; refresh
it with invoke catalog-refresh, which prints the maintenance runbook
(src/llmbroker/presets/freetier-refresh-prompt.md).
Reports:
Created with cookiecutter using template
1 maintainer
Metadata
1 maintainer
Release files for llmbroker 1.11.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| llmbroker-1.11.0.tar.gz | 673.9 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| llmbroker-1.11.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 817.6 kB
Release files / llmbroker-1.11.0.tar.gz
| Download URL | llmbroker-1.11.0.tar.gz |
|---|---|
| Size | 673.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
a92a207b96d75aafb6827452d71a439b5995d94f811133b82a25cbc3a99eac57
|
|
BLAKE2b-256 checksum How to use checksums |
dce65017f6a24c132d2b9280c8050ddfd82c0803217ca69ea7e42cf9848f2c92
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Release files / llmbroker-1.11.0-py3-none-any.whl
| Download URL | llmbroker-1.11.0-py3-none-any.whl |
|---|---|
| Size | 143.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
be0383f23ee439cee2d1a0509ae974fadc227ac0311d12ba908f8160b45d40dd
|
|
BLAKE2b-256 checksum How to use checksums |
06e2c28f98c85f9506458dbd2725476479270867add69a6e626a0d440fb87374
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
