SmolLLM

A minimal Python library for interacting with various LLM providers, featuring automatic API key load balancing and streaming responses.

Installation

pip install smolllm
uv add "smolllm @ ../smolllm"

Quick Start

import asyncio
from smolllm import ask_llm

async def main():
    response = await ask_llm(
        "Say hello world",
        model="gemini/gemini-2.0-flash"
    )
    print(response)

if __name__ == "__main__":
    asyncio.run(main())

Reasoning Effort

For reasoning-capable models, pass reasoning_effort through to the provider. Useful for Ollama reasoning models where omitting it can make replies much slower.

response = await ask_llm(
    "Reply with one word.",
    model="ollama/qwen3.5:0.8b",
    base_url="http://rocry-ubuntu.local:11434/v1",
    reasoning_effort="none",
)

Provider Configuration

Format: provider/model_name (e.g., openai/gpt-4, gemini/gemini-2.0-flash)

A bare model name (no /) targets an endpoint directly — pass base_url and api_key explicitly (no env fallback):

response = await ask_llm(
    "hi",
    model="gpt-4",
    base_url="https://my-proxy.example/v1",
    api_key="sk-xxx",
)

API Keys

The library looks for API keys in environment variables following the pattern: {PROVIDER}_API_KEY

Example:

# .env
OPENAI_API_KEY=sk-xxx
GEMINI_API_KEY=key1,key2  # Multiple keys supported

Custom Base URLs

Override default API endpoints using: {PROVIDER}_BASE_URL

Example:

OPENAI_BASE_URL=https://custom.openai.com/v1
OLLAMA_BASE_URL=http://localhost:11434/v1

Advanced Configuration

You can combine multiple keys and base URLs in several ways:

  1. One key with multiple base URLs:
OLLAMA_API_KEY=ollama
OLLAMA_BASE_URL=http://localhost:11434/v1,http://other-server:11434/v1
  1. Multiple keys with one base URL:
GEMINI_API_KEY=key1,key2
GEMINI_BASE_URL=https://api.gemini.com/v1
  1. Paired keys and base URLs:
# Must have equal number of keys and URLs
# The library will randomly select matching pairs
GEMINI_API_KEY=key1,key2
GEMINI_BASE_URL=https://api.gemini.com/v1,https://api.gemini.com/v2

Environment Setup Best Practices

When using SmolLLM in your project, you should handle environment variables at your application level:

  1. Create a .env file:
# .env
OPENAI_API_KEY=sk-xxx
GEMINI_API_KEY=xxx,xxx2
ANTHROPIC_API_KEY=sk-xxx
  1. Load the file at the application/process boundary:
uv run --env-file .env app.py

Tips

  • Keep sensitive API keys in .env (add to .gitignore)
  • Create .env.example for documentation
  • For production, consider using your platform's secret management system
  • When using multiple keys, separate with commas (no spaces)

Project links

Data verified by PyPI on Jul 30, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.10.0)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for rocry from gravatar.com rocry

Credits

Author: RoCry

GitHub Statistics

Data verified by PyPI on Jul 30, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License

MIT License (MIT)

Requires

Python >=3.13

Provides Extra

dev test

Tags

ai anthropic gemini llm openai

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

smolllm-0.9.0.tar.gz (66.8 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

smolllm-0.9.0-py3-none-any.whl (28.4 kB view details)

Uploaded Python 3

File details

Details for the file smolllm-0.9.0.tar.gz.

File metadata

  • Download URL: smolllm-0.9.0.tar.gz
  • Upload date:
  • Size: 66.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for smolllm-0.9.0.tar.gz
Algorithm Hash digest
SHA256 3745595f166d0669e173c2c25f8c728d13073c115990bffb70fef21f18db9776
MD5 c06dae92fe6dd82848d75c2190c83cdd
BLAKE2b-256 88bb92f8f93ab378b453465ce77be8129b17ca015604317812f9d95a98a3bdb7

See more details on using hashes here.

Provenance

The following attestation bundles were made for smolllm-0.9.0.tar.gz:

Publisher: release.yml on RoCry/smolllm

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file smolllm-0.9.0-py3-none-any.whl.

File metadata

  • Download URL: smolllm-0.9.0-py3-none-any.whl
  • Upload date:
  • Size: 28.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for smolllm-0.9.0-py3-none-any.whl
Algorithm Hash digest
SHA256 0d189497cbdc25360ed409b6d5e4846b75f37313e3f36cf7db7f9c3ed5622b1d
MD5 7fcb8fb75f6cf0d5efd9ce44752d3ff4
BLAKE2b-256 338d029ce5a59453318ebcdc59fea2644778edf11e2c9f7d09e0ca3ed923094e

See more details on using hashes here.

Provenance

The following attestation bundles were made for smolllm-0.9.0-py3-none-any.whl:

Publisher: release.yml on RoCry/smolllm

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page