splunk-otel-util-genai 0.1.9

OpenTelemetry GenAI Utils

Minimal Overview

Utility function to provide APIs and data types to ease instrumentation of Generative AI workloads using OpenTelemetry semantic conventions.

Example usage for LLM Invocation.

from opentelemetry.util.genai.handler import get_telemetry_handler
handler = get_telemetry_handler()
user_input = "Hello"
inv = LLMInvocation(request_model="gpt-5-nano", provider="openai",
  input_messages=[InputMessage(role="user", parts=[Text(user_input)])])
handler.start_llm(inv)
# your code which actuall invokes llm here
# response = client.chat.completions.create(...)
# ....
inv.output_messages = [OutputMessage(role="assistant", parts=[Text("Hi!")], finish_reason="stop")]
handler.stop_llm(inv)

See the example in examples/agentic_example.py for a full agent + LLM invocation flow.

Concurrent Evaluation Mode

For high-throughput LLM-as-a-Judge evaluations, enable concurrent processing:

# Enable concurrent evaluation with 4 workers
export OTEL_INSTRUMENTATION_GENAI_EVALS_CONCURRENT=true
export OTEL_INSTRUMENTATION_GENAI_EVALS_WORKERS=4

# Optional: Set bounded queue for backpressure
export OTEL_INSTRUMENTATION_GENAI_EVALS_QUEUE_SIZE=100

Configuration options:

• OTEL_INSTRUMENTATION_GENAI_EVALS_CONCURRENT - Enable concurrent mode (default: false)

• OTEL_INSTRUMENTATION_GENAI_EVALS_WORKERS - Number of worker threads (default: 4)

• OTEL_INSTRUMENTATION_GENAI_EVALS_QUEUE_SIZE - Bounded queue size, 0 for unbounded (default: 0)

Concurrent mode processes multiple evaluations in parallel, significantly improving throughput when using LLM-as-a-Judge metrics like DeepEval’s Bias, Toxicity, and Answer Relevancy.

Metrics

This library emits histogram metrics with explicit bucket boundaries per OpenTelemetry GenAI semantic conventions:

Duration Metrics (unit: seconds)

• gen_ai.client.operation.duration - Duration of GenAI client operations

• gen_ai.workflow.duration - Duration of GenAI workflows

• gen_ai.agent.duration - Duration of agent operations

Bucket boundaries: [0.01, 0.02, 0.04, 0.08, 0.16, 0.32, 0.64, 1.28, 2.56, 5.12, 10.24, 20.48, 40.96, 81.92]

Token Usage Metrics (unit: tokens)

• gen_ai.client.token.usage - Number of input and output tokens used

Bucket boundaries: [1, 4, 16, 64, 256, 1024, 4096, 16384, 65536, 262144, 1048576, 4194304, 16777216, 67108864]

Evaluation Metrics (unit: score 0-1)

• gen_ai.evaluation.score - GenAI evaluation score (default, when OTEL_INSTRUMENTATION_GENAI_EVALS_USE_SINGLE_METRIC is unset or true)

• gen_ai.evaluation.<name> - Individual evaluation metrics (when OTEL_INSTRUMENTATION_GENAI_EVALS_USE_SINGLE_METRIC=false)

Bucket boundaries: [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]

Further Documentation

For architecture, design rationale, and broader usage patterns please consult: * Core concepts, high-level usage and setup * README.packages.architecture.md – extensibility architecture & emitter pipeline design.

Those documents cover configuration (environment variables, content capture modes, evaluation emission, extensibility via entry points) and release/stability notes.

Support & Stability

GenAI semantic conventions are incubating.

License

Apache 2.0 (see LICENSE).