OpenTelemetry smolagents Instrumentation

This library provides OpenTelemetry instrumentation for smolagents. It wraps the model classes that run inference in your own process and emits a GenAI semantic-convention chat span and the matching metrics through opentelemetry-util-genai:

• TransformersModel

• VLLMModel

• MLXModel

The API-backed model classes are not instrumented here. Each one calls a client library that carries its own instrumentation. Emitting a span at the smolagents layer as well would produce two chat spans for one model call, and would count the token-usage and duration metrics twice. Install the instrumentation for the client library instead:

TransformersModel is the only instrumented class with a generate_stream. A streamed call gets a chat span that stays open until the caller drains the deltas. This covers both stream_outputs=True on an agent and a direct generate_stream call. The span carries gen_ai.request.stream, and the call also records the gen_ai.client.operation.time_to_first_chunk and gen_ai.client.operation.time_per_output_chunk metrics.

Known gaps:

• A managed agent called from the generated code of a CodeAgent runs in a worker thread of the local Python executor. That worker does not carry the calling context, so the managed agent’s invoke_agent span starts a trace of its own instead of nesting under the manager’s span.

• A subclass that inherits generate or generate_stream from one of the three classes above is instrumented. A subclass that overrides one is not: the override shadows the patched method, so the call produces no chat span.

• A chat span reports no gen_ai.response.id, no gen_ai.response.model, no gen_ai.response.finish_reasons and no server.address. A runtime in this process returns the generated text and the token counts, nothing more. It also listens on no socket.

• Tool calls (execute_tool) are not instrumented yet.

pip install opentelemetry-instrumentation-genai-smolagents

from opentelemetry.instrumentation.genai.smolagents import (
    SmolagentsInstrumentor,
)
from smolagents import CodeAgent, TransformersModel

SmolagentsInstrumentor().instrument()

model = TransformersModel(model_id="HuggingFaceTB/SmolLM2-135M-Instruct")
agent = CodeAgent(tools=[], model=model)
agent.run("How many seconds are in a week?")

export OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=SPAN_AND_EVENT

export OTEL_INSTRUMENTATION_GENAI_COMPLETION_HOOK=upload
export OTEL_INSTRUMENTATION_GENAI_UPLOAD_BASE_PATH=/path/to/prompts  # or gs://my_bucket

from opentelemetry.instrumentation.genai.smolagents import (
    SmolagentsInstrumentor,
)

SmolagentsInstrumentor().instrument(completion_hook=my_hook)