octoryn-llm

Official Python client for the independent Octoryn Router.

Install

pip install octoryn-llm

Quickstart

import os
from octoryn import Octoryn

client = Octoryn(api_key=os.environ["OCTORYN_API_KEY"])
result = client.chat.completions.create(
    model="openai/gpt-4.1-mini",
    messages=[{"role": "user", "content": "Hello"}],
)
print(result.choices[0].message.content)

The default base URL is https://api.octoryn.dev/v1. Service credentials need the Router route scope.

Supported wire resources

• Inference: chat.completions, completions, responses, embeddings, moderations, and rerank.
• Catalogue: models and providers.
• Management: keys, governance, route.simulate, and operations.

Management resources require their documented Router scopes and are kept separate from inference resources. Key creation and revocation require an explicit idempotency key; revocation also requires a reason code.

The production Router does not expose the former Gateway audio, image, video, realtime, BYOK, usage-run, audit-run, agent, document, memory, knowledge, or skills endpoints. Those methods were removed in v3 instead of remaining as methods that always return 404. The octoryn.openai_compat alias raises NotImplementedError for absent OpenAI surfaces.

Framework integrations are optional and use Chat Completions only:

pip install "octoryn-llm[langchain]"
pip install "octoryn-llm[llamaindex]"

The checked-in production method/path manifest is ../../contracts/router-api-v1.json. Run python3 ../../scripts/sdk/check_router_contract.py --live before release.