LLM Tracekit - OpenAI

OpenTelemetry instrumentation for OpenAI, designed to simplify LLM application development and production tracing and debugging.

Installation

OpenAI

pip install "llm-tracekit-openai"

Usage

This section describes how to set up instrumentation. The examples will use the OpenAI instrumentation, but the usage is similar for all instrumentations.

Setting up tracing

You can use the setup_export_to_coralogix function to setup tracing and export traces to Coralogix

from llm_tracekit.openai import setup_export_to_coralogix

setup_export_to_coralogix(
    service_name="ai-service",
    application_name="ai-application",
    subsystem_name="ai-subsystem",
    capture_content=True,
)

Alternatively, you can set up tracing manually:

from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import SERVICE_NAME, Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor

tracer_provider = TracerProvider(
    resource=Resource.create({SERVICE_NAME: "ai-service"}),
)
exporter = OTLPSpanExporter()
span_processor = SimpleSpanProcessor(exporter)
tracer_provider.add_span_processor(span_processor)
trace.set_tracer_provider(tracer_provider)

Activation

To instrument all clients, call the instrument method

from llm_tracekit import OpenAIInstrumentor

OpenAIInstrumentor().instrument()

Enabling message content

Message content such as the contents of the prompt, completion, function arguments and return values are not captured by default. To capture message content as span attributes, do one of the following:

• Pass capture_content=True when calling setup_export_to_coralogix
• Set the environment variable OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT to true

Most Coralogix AI evaluations will not work without message contents, so it is highly recommended to enable capturing.

Uninstrument

To uninstrument clients, call the uninstrument method:

OpenAIInstrumentor().uninstrument()

Full Example

from llm_tracekit.openai import OpenAIInstrumentor, setup_export_to_coralogix
from openai import OpenAI

# Optional: Configure sending spans to Coralogix
# Reads Coralogix connection details from the following environment variables:
# - CX_TOKEN
# - CX_ENDPOINT
setup_export_to_coralogix(
    service_name="ai-service",
    application_name="ai-application",
    subsystem_name="ai-subsystem",
    capture_content=True,
)

# Activate instrumentation
OpenAIInstrumentor().instrument()

# Example OpenAI Usage
client = OpenAI()
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "user", "content": "Write a short poem on open telemetry."},
    ],
)

Responses API

Example Response API call

from openai import OpenAI

client = OpenAI()
response = client.responses.create(
    model="gpt-4o-mini",
    input="Write a haiku about OpenTelemetry.",
)

from openai import OpenAI

client = OpenAI()
with client.responses.stream(
    model="gpt-4o-mini",
    input="Write a haiku about OpenTelemetry.",
) as stream:
    for event in stream:
        pass
    response = stream.get_final_response()

Embeddings Example

from openai import OpenAI

client = OpenAI()
response = client.embeddings.create(
    model="text-embedding-3-small",
    input="What is machine learning?",
)
print(f"Embedding dimensions: {len(response.data[0].embedding)}")

Changes from OpenTelemetry

General

• The user parameter in the OpenAI Chat Completions API is now recorded in the span as the gen_ai.request.user attribute
• User prompts and model responses are captured as span attributes instead of log events (see Semantic Conventions below)

Semantic Conventions

Function spans

These spans represent the execution of a tool (a Python function).

Enriched LLM call spans

These attributes are added to the existing span to link LLM calls back to the responsible agent.