middleware-llmobs

LLM Observability for the Middleware SaaS platform, built on OpenInference and OpenTelemetry.

middleware-llmobs provides a one-line register() that configures an OpenTelemetry TracerProvider to export OpenInference traces to your Middleware collector over OTLP/HTTP.

Transport: Only HTTP/protobuf is supported. gRPC is intentionally disabled — requesting it raises NotImplementedError. (The gRPC code path is retained in the source for future use.)

Install

pip install middleware-llmobs

Quickstart

Set the standard OTLP environment variables for your Middleware account:

export OTEL_EXPORTER_OTLP_ENDPOINT="https://uid.middleware.io:443"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=<your-key>"
export OTEL_SERVICE_NAME="my-llm-app"

Then register and auto-instrument every installed OpenInference library:

from middleware.llmobs import register

providers = register(auto_instrument=True)  # batch + HTTP by default
# providers.tracer / providers.logger / providers.meter — the wired OTel providers.

Explicit configuration

from middleware.llmobs import register

providers = register(
    endpoint="https://uid.middleware.io:443",
    headers={"Authorization": "<your-key>"},
    service_name="my-llm-app",
)

Manual instrumentor wiring with GenAI semantic conventions

from middleware.llmobs import register
from openinference.instrumentation import TraceConfig
from openinference.instrumentation.openai import OpenAIInstrumentor

providers = register(service_name="my-llm-app")

OpenAIInstrumentor().instrument(
    tracer_provider=providers.tracer,
    config=TraceConfig(enable_genai_semconv=True),
)

When auto_instrument=True, register() passes TraceConfig(enable_genai_semconv=True) to each instrumentor automatically (falling back gracefully for instrumentors that don't accept a config argument).

Environment variables

License

Apache-2.0