roboz-endpoints

Build a typed catalogue of the providers and models your application uses. The bundled OpenRouter, Cerebras, and Groq catalogues are examples to start from. Add your own providers and models through the inventory commands below; your catalogue lives in your project and requires no changes to this library.

Current limitation

Only providers with an OpenAI-compatible API are currently supported. Adding a provider to the catalogue does not add support for a different API protocol.

Requires Python 3.13+.

uv add 'roboz-endpoints[openai]'

Add your own provider and model

Run these commands from your project root:

# 1. Export the examples into your application's model_catalogue package.
uv run python -m roboz_endpoints inventory export

# 2. Edit models.json: add, remove, or change providers and models.

# 3. Generate model_catalogue/providers.py.
uv run python -m roboz_endpoints inventory import

For a src-layout project named my-app, this creates:

src/my_app/model_catalogue/
├── __init__.py
├── models.json                # edit this
└── providers.py               # generated by inventory import

For example, add this entry under providers in models.json before running the import command:

"my_service": {
  "base_url": "https://models.example.com/v1",
  "api_key_env": "MY_SERVICE_API_KEY",
  "models": {
    "chat": {
      "model_id": "my-chat-model",
      "endpoint_type": "llm",
      "max_context_tokens": 128000
    }
  }
}

The catalogue can then be imported by any function or module run from the project environment:

from roboz.llm import LLMEndpoint
from my_app.model_catalogue.providers import my_service


def get_chat_endpoint() -> LLMEndpoint:
    return my_service.chat

endpoint = get_chat_endpoint()
print(endpoint.model_name)          # my-chat-model
print(endpoint.max_context_tokens)  # 128000

# Use this instance as Agent(..., agent_endpoint=endpoint).

The JSON keys my_service and chat become Python names; model_id is the identifier sent to the provider. Add more entries under models for another model, or under providers for another service. Use valid Python identifiers for those keys.

Run that code with uv run python your_script.py. It only verifies the generated configuration; it does not contact the provider. Before making a real model call, use the provider's actual URL, model ID, and context limit, and set MY_SERVICE_API_KEY. The JSON stores the variable name, not the secret.

For flat layouts, arbitrary catalogue names, custom generated-module names, updates, reset, and recovery, see the project catalogue guide.

Existing catalogues keep their locations when you pass explicit --path and --output; see upgrading an existing catalogue.

License

Licensed under the Apache License 2.0.