orbitrage 0.6.0

One-line observability + intelligent LLM routing for Python agents.

orbitrage

One LLM gateway. Router + observability. One line.

Orbitrage is the proxy between your code and the model providers. Every request flows through it — so it sees the model you asked for, the model it routed to, the input messages, the output, tokens, cost, and latency. Pointing your OpenAI SDK at https://api.orbitrage.ai/v1 is the entire integration; this package is a 5-line convenience wrapper that does it for you and tags every call with a trace + user id so the dashboard groups them properly.

import orbitrage
orbitrage.init("orb_xxx_yyy")           # 1 line of setup
                                         #
from openai import OpenAI                # 2 lines of usage — the SDK auto-points
client = OpenAI()                        #   at the Orbitrage gateway and adds
                                         #   trace headers to every request.
client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "hi"}],
)

Open https://app.orbitrage.ai/workflows — every call shows up with model, tokens, cost, latency, and the full input/output.

Install

pip install orbitrage

The only runtime dependency is the Python standard library. openai and anthropic are optional peers — install whichever clients you use.

What init() does

1. Generates a stable trace id for this process. Every request from this process carries it as x-orbitrage-run-id, so the dashboard groups all your agent's LLM calls into a single run.
2. Patches the OpenAI and Anthropic client constructors to default base_url to https://api.orbitrage.ai/v1 and add the trace headers via default_headers. If you already set base_url or api_key explicitly, your value wins.
3. Sets the OPENAI_BASE_URL env var as a safety net for code that constructs the client without any args.

That's all. No background threads, no span processors, no OTLP exports. The gateway sees every byte of the request and writes the routing decision, the upstream response, tokens, cost, and latency directly to your project's data store. The proxy is the single source of truth.

Per-end-user graphs

If you're a B2B service handling traffic for many end-users (your customers' users), pass user_id so the dashboard can partition every call inside one workflow:

orbitrage.init("orb_xxx", user_id=current_user.id)

Every subsequent LLM call is tagged with x-orbitrage-end-user-id. The dashboard uses it to build per-user flow graphs, cost breakdowns, and churn signals.

For long-running servers that switch end-users between requests, call orbitrage.set_user(user_id) at the start of each request. The next OpenAI or Anthropic client constructed after that picks up the new id.

Bring your own provider key (BYOK)

Save your OpenAI / Anthropic / Google / Groq key on the Models page. For models that match that provider, Orbitrage forwards the request to the provider with your key — your account is billed, our pooled credits are untouched. Routing decisions still appear in your dashboard with byok=true so you can see the split.

Forcing a specific model

Pass the model id verbatim — Orbitrage routes by explicit pin when the caller names a model:

client.chat.completions.create(
    model="claude-sonnet-4-6",     # explicit pin, no scoring
    messages=[...],
)

To let the router pick the cheapest model that meets the prompt's difficulty, pass the alias auto (or any of router, default, orbitrage).

Backward compatibility

Functions kept as no-op shims for code written against v0.4:

@orbitrage.workflow("checkout")
def run(): ...

@orbitrage.task("plan")
def plan(): ...

orbitrage.flush()       # no-op — the gateway persists synchronously
orbitrage.shutdown()    # no-op

These return the wrapped function untouched. The router captures the same data regardless of decorator markup; the decorators exist so v0.4 user code keeps importing without errors.

How the workflow is determined

Every API key is minted for one workflow from the dashboard. The workflow your calls land under is whichever workflow your key belongs to — no app_name argument required.

License

Apache-2.0