keel-llm-otel

OpenTelemetry metrics + span events for keel-llm-reliability. A pure external wrapper around ResilientClient — emits signals from the visible-degradation trail after each call. keel-llm-reliability itself stays observability-free.

Part of the Keel toolkit. Composes cleanly: drop in over any ResilientClient instance, configure your OTel SDK however you like, see your multi-model LLM calls light up in Jaeger / Datadog / Honeycomb / Grafana / wherever.

Is this for you?

Adopt when — you run keel-llm-reliability in production with OpenTelemetry already in your stack and want per-attempt metrics + span events flowing into your existing tracing/metrics backend without writing the bridge yourself. Skip when — you're not on OTel; you're prototyping (the visible Attempt trail in return values is plenty for development); you want a different backend (write your own thin wrapper over ResilientClient — the pattern here is ~80 LOC).

Install

pip install keel-llm-otel
# you bring your own OTel SDK + exporter, e.g.:
#   pip install opentelemetry-sdk opentelemetry-exporter-otlp

Four seamless paths — pick one

Path 0 — One-line setup (fastest trial) — [starter] extra

Production OTel SDK setup is famously verbose. [starter] bundles the SDK + an OTLP/gRPC exporter + a setup() helper that does the standard wiring in one line:

pip install 'keel-llm-otel[starter]'        # SDK + OTLP exporter + auto-instrumentation

from keel_llm_otel.starter import setup
setup()    # SDK configured, OTLP export to localhost:4317, auto-instrumentor active.

# That's it. Your existing ResilientClient code emits metrics + span events.

Env-driven defaults (override with kwargs to setup()):

• OTEL_EXPORTER_OTLP_ENDPOINT → default http://localhost:4317 (SigNoz / Jaeger / your collector)
• OTEL_SERVICE_NAME → default keel-app

Production at scale typically configures the SDK its own way (custom resource attributes, sampling, batching tuned to your collector). [starter] is a sane default and a substitute for the boilerplate during trial / dev — not a substitute for production-grade SDK config.

Path 1 — Full distributed tracing — [full] extra

One install, complete distributed tracing of LLM calls — reliability-layer metrics + span events and per-provider HTTP-level child spans with traceparent propagated to providers. Auto-instrumentor + opentelemetry-instrumentation-httpx activate together.

pip install 'keel-llm-otel[full]'                # everything wired
opentelemetry-instrument python yourapp.py        # discovers + activates automatically

Why [full] is recommended: distributed tracing requires HTTP-level instrumentation to inject traceparent headers into outgoing provider calls. Installing only keel-llm-otel[auto] gives correct reliability-layer metrics + events but silently omits per-call child spans and trace propagation — [full] eliminates that failure mode.

Path 1b — Auto-instrument only (reliability layer, no HTTP spans) — [auto] extra

For consumers who already use opentelemetry-instrumentation-httpx elsewhere, or who want the lighter dep set (no wrapt transitively from httpx instrumentation):

pip install 'keel-llm-otel[auto]'                # adds opentelemetry-instrumentation + wrapt
opentelemetry-instrument python yourapp.py        # discovers + activates automatically

Or programmatically — one line at startup:

from keel_llm_otel import OTelInstrumentor
OTelInstrumentor().instrument()    # call once; ResilientClient is patched globally

Your existing code is unchanged:

from keel_llm_reliability import ResilientClient, Request
client = ResilientClient([...])
result = await client.failover(req)   # metrics + span events emitted automatically

Path 2 — Drop-in subclass (one-word code change) — lean install

pip install keel-llm-otel (no extras needed) gets you a ResilientClient subclass with emission built in. When you want explicit, per-instance control (multi-tenant meters, test isolation, custom Tracer):

- from keel_llm_reliability import ResilientClient
+ from keel_llm_otel import OTelObservableClient as ResilientClient

Same constructor, same methods, same return types — OTelObservableClient is a ResilientClient:

import asyncio
from keel_llm_otel import OTelObservableClient as ResilientClient
from keel_llm_adapter_openai import OpenAIAdapter
from keel_llm_reliability import Request
from keel_llm_protocol import user

client = ResilientClient([                 # same call as before
    OpenAIAdapter(model="llama-3.3-70b-versatile", api_key="gsk_…",
                  base_url="https://api.groq.com/openai/v1", provider="groq"),
    OpenAIAdapter(model="llama-3.1-8b", base_url="http://localhost:11434/v1",
                  provider="local"),
])

async def main() -> None:
    result = await client.failover(Request(messages=[user("Hello.")]))
    # ↑ identical behaviour; metrics + span events emitted as a side-effect.

asyncio.run(main())

Both paths produce identical signals; both use OTel's no-op meter/tracer when nothing is configured (zero overhead, zero failure).

Path 3 — Direct emission (decoupled from keel-llm-reliability's orchestrator)

If you have your own multi-model orchestrator (in-tree fan-out / failover code), you can still emit on the same OTel vocabulary — Signals.emit(...) accepts any iterable of AttemptLike (a structural Protocol: anything with model_key / outcome / latency_ms / error). No keel-llm-reliability adoption required.

from keel_llm_otel import Signals
# Build your own attempt records (any shape with the four fields above):
attempts = [
    MyAttempt(model_key="gpt-4o", outcome="success", latency_ms=120, error=None),
    MyAttempt(model_key="claude-sonnet", outcome="deferred_backpressure",
              latency_ms=45, error=MyError(category="backpressure")),
]
Signals().emit(strategy="my_orchestrator", attempts=attempts,
               succeeded=True, degraded=True)

Useful when adopting Keel's full orchestrator isn't the right move (you have working in-tree machinery), but you still want unified OTel emission across consumers in your portfolio.

What gets emitted

Configure your OTel SDK (exporters, resource attributes, sampling) however you normally would — this package only produces the signals.

Metrics

(Counter names follow modern OTel convention — no _total suffix; the Prometheus exporter adds it automatically. The model-name label uses OTel's incubating GenAI convention gen_ai.request.model so reliability-layer dispositions can be joined with adapter-level gen_ai.* metrics in cross-layer queries.)

strategy is "failover" or "fan_out". outcome is one of success / preempted_open / preempted_limited / deferred_backpressure / empty / failed. error_category is the failed attempt's error.category (backpressure / transient / terminal), or "none" when the attempt didn't carry an error.

Span events

If the caller has a current span (the standard pattern — wrap your reliability call in your own span), each Attempt becomes a keel.attempt event on that span with attributes keel.strategy, gen_ai.request.model, keel.outcome, keel.latency_ms, keel.error_category:

from opentelemetry import trace

tracer = trace.get_tracer(__name__)

with tracer.start_as_current_span("llm.call") as span:
    result = await client.failover(req)
    # span now carries one `keel.attempt` event per provider interaction —
    # visible in your tracing UI alongside whatever else you put on it.

Design notes

• Pure wrapper, no coupling. keel-llm-reliability doesn't depend on OTel; this package wraps it externally. Want a different backend (Prometheus, Datadog APM, structured-log emitter)? Write your own ~80-LOC wrapper over ResilientClient — the Attempt trail is the contract.
• Post-call emission only. v0 emits signals after the wrapped call returns, using latency_ms from each Attempt. No new spans are started. Proper async span-context propagation into provider HTTP calls belongs in adapter-level HTTP instrumentation (e.g. opentelemetry-instrumentation-httpx), not in a reliability-layer wrapper.
• Emission errors are swallowed. Observability must never break the underlying call. If a meter or tracer raises, the wrapped call's result is still returned.
• No global-state assumptions. Pass a specific Meter / Tracer for tests / multi-tenant; or omit and it picks up the global providers your SDK configured.

Status

0.1.0 — 0.x while the API stabilizes through year one (breaking changes possible at minor bumps, documented in the CHANGELOG; pin exact versions).

The Keel toolkit

Composable, vendor-neutral LLM reliability libraries on PyPI: keel-llm-reliability · keel-llm-protocol · keel-llm-adapter-openai · keel-llm-adapter-anthropic · keel-llm-adapter-google · keel-circuit-breaker · keel-llm-otel

MIT licensed.