Skip to main content
Pre-release

This release is a pre-release and may not be stable for production use.

Llimona

Llimona is an open and modular Python framework for building production-ready LLM gateways. It provides OpenAI-compatible APIs, provider-aware routing, and an extensible plugin model for integrating multiple backends behind a single interface.

By keeping providers as addons, Llimona stays lightweight at its core while enabling deployments to include only the integrations, policies, and observability components they actually need.

Key Features

  • OpenAI-compatible service interfaces (currently Responses and Models).
  • Provider routing using the provider_name/model_name naming convention.
  • Addon-based extensibility through Python entry points (llimona.addon).
  • Typed YAML configuration with Pydantic validation.
  • Request Context propagation with actor/origin metadata, constraints, and sub-context trees.
  • Sensor support for metrics such as request counters and elapsed time, making request execution observable.

Architecture

Architecture documentation

Requirements

  • Python >= 3.14
  • uv (recommended)

Installation

Install dependencies for local development

uv sync

Install the core package

uv pip install .

Install an addon package

uv pip install ./addons/llimona_azure_openai

Quick Start

1) Create an app config

Example (test_config/app.yaml):

provider_addons:
  - azure_openai
provider_loaders:
  - type: autodiscovery_dirs
    src: !path .

2) Create a provider directory with provider.yaml

Example (example_config/azure_1/provider.yaml):

type: azure_openai
name: azure_1
display_name: Azure Example 1
owner_id: 444444-222-333-222 # Not used, just for future purposes
base_url: !envvar AZURE_OPENAI_1_BASE_URL
credentials:
  api_key: !envvar AZURE_OPENAI_1_API_KEY
services:
- type: openai_responses
- type: openai_models
models:
- name: gpt-4o-mini
  allowed_services:
  - openai_responses

3) Run a request

uv run llimona app --config-file example_config/app.yaml openai responses create azure_1/gpt-4o-mini "Hello" --stream

4) Observe sensor metrics

After the request completes, Llimona prints sensor values that make execution observable:

Sensor value: elapsed_time=0.606314 (Elapsed time of the request.)
Sensor value: request_count=1 (Number of requests being processed for the sensor request_count.)
Sensor value: request_per_unit_of_time=1 (Number of requests in the last 0:01:00.)
Sensor value: request_per_window_of_time=1 (Number of requests until the next reset.)

CLI Usage

Top-level help

llimona --help

List discovered addons

llimona addons

Run commands with an app config

llimona app --config-file <path-to-app.yaml> <command>

Providers

# list all providers
llimona app --config-file <cfg> providers

# inspect one provider
llimona app --config-file <cfg> providers <provider_name>

# list models in one provider
llimona app --config-file <cfg> providers <provider_name> models

OpenAI-compatible interface commands

# create a response
llimona app --config-file <cfg> openai responses create <provider>/<model> "Prompt"

# streaming response
llimona app --config-file <cfg> openai responses create <provider>/<model> "Prompt" --stream

# list models (global or filtered by provider)
llimona app --config-file <cfg> openai models list
llimona app --config-file <cfg> openai models list <provider_name>

Configuration Overview

The app configuration supports these top-level fields:

  • provider_addons: provider addons to register.
  • provider_loader_addons: provider-loader addons to register.
  • sensor_addons: sensor addons to register.
  • id_builder: optional ID builder configuration.
  • provider_loaders: loader definitions.

Built-in provider loader:

  • autodiscovery_dirs: scans child directories under src, reads provider.yaml, and optionally merges definitions from models/*.yaml, services/*.yaml, and sensors/*.yaml.

Architecture Summary

Llimona receives OpenAI-compatible requests, decomposes model IDs, routes to the appropriate provider, and maps provider-specific responses back to interface models.

Every call flows through a Context object, which can carry:

  • action metadata (provider, service, service_action, model)
  • actor and origin information
  • conversation metadata
  • constraints
  • collected sensor values

Routing strategies can create sub-contexts, enabling per-branch observability and post-execution failure inspection.

Sensors make the platform observable by exposing execution metrics across the full request context tree.

For full technical details, see docs/arch.md.

Addons in This Repository

  • addons/llimona_azure_openai: Azure OpenAI provider addon.
  • addons/llimona_smart_provider: smart/virtual provider routing addon.

Development

Install development tools

uv sync --group dev

Run tests

uv run pytest

Lint and format

uv run ruff check .
uv run ruff format .

Branching and Versioning

The repository follows a GitFlow-like model with:

  • main as the default integration branch
  • feat/*, fix/*, and chore/* working branches
  • squash-merge pull requests
  • SemVer/PEP 440 release semantics

See branching model document for the complete policy.

Security Notes

  • Do not commit real API keys or secrets in provider files.
  • Inject credentials at runtime through your deployment environment.

License

This project is licensed under the GNU AFFERO GENERAL PUBLIC LICENSE. See LICENSE for details.

Metadata

Release files for Llimona 0.1.0.dev2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for Llimona 0.1.0.dev2
File Size Uploaded Provenance Actions
llimona-0.1.0.dev2.tar.gz 44.9 kB With provenance Details for llimona-0.1.0.dev2.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for Llimona 0.1.0.dev2
File Interpreter ABI Platform Actions
llimona-0.1.0.dev2-py3-none-any.whl Python 3 none any Details for llimona-0.1.0.dev2-py3-none-any.whl

Total release size: 99.4 kB

Release files / llimona-0.1.0.dev2.tar.gz

Download URL llimona-0.1.0.dev2.tar.gz
Size 44.9 kB
Tags Source
SHA-256 checksum
How to use checksums
c04c1bfa0259af39d56104c48a08ead7bdbe818f82805a02a1c69a1bf0587428
BLAKE2b-256 checksum
How to use checksums
99f8214b2b4364e996095456b3f63749f6460f8823cfd89311e65676d3471242
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.12

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Apr 12, 2026.

Transparency log

Release files / llimona-0.1.0.dev2-py3-none-any.whl

Download URL llimona-0.1.0.dev2-py3-none-any.whl
Size 54.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
6189385b16e9d0bef184d6fe61dd0b387edc90308d51c59915313dd906c340f8
BLAKE2b-256 checksum
How to use checksums
5b3be8ba672f0c045aefe837475d7fe4b12234c253faccddf87ac58f3b7638d2
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.12

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Apr 12, 2026.

Transparency log

Security for Llimona 0.1.0.dev2

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for llimona-0.1.0.dev2.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Apr 12, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release
Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page