traceAI-semantic-kernel

Future AGI tracing for Semantic Kernel for Python, built on Semantic Kernel's own OpenTelemetry diagnostics.

Experimental upstream diagnostics. Python only. This is not the Java package (java/traceai-java-semantic-kernel), not .NET Semantic Kernel, and not Microsoft Agent Framework (use traceAI-agent-framework for that). Semantic Kernel marks its GenAI diagnostics @experimental, so the span names and keys below can change in a future Semantic Kernel release.

What it does

SemanticKernelInstrumentor().instrument():

1. Turns on Semantic Kernel's GenAI diagnostics in process. You do not export SEMANTICKERNEL_EXPERIMENTAL_GENAI_ENABLE_OTEL_DIAGNOSTICS. Semantic Kernel builds its ModelDiagnosticSettings objects when its modules are imported, so setting the environment variable afterwards has no effect; instrument() sets the fields on those objects instead. Calling it before or after you build the kernel both work.
2. Points Semantic Kernel's module-level tracers at the tracer provider you pass. Semantic Kernel creates them from the global provider, and fi_instrumentation.register() does not set the global provider by default.
3. Installs a mapping-only span processor ahead of the exporter: span kind, provider alias, total tokens, session, content default.

It does not wrap Kernel.invoke, KernelFunction.invoke or any connector method, opens no spans of its own, and does not depend on wrapt. Semantic Kernel creates every span you see.

Install

pip install traceAI-semantic-kernel

semantic-kernel (MIT) is a dependency floor only (>=1.38.0,<2); it is not vendored.

Use

export FI_API_KEY="YOUR_API_KEY"
export FI_SECRET_KEY="YOUR_SECRET_KEY"

from fi_instrumentation import register, using_session
from fi_instrumentation.fi_types import ProjectType
from traceai_semantic_kernel import SemanticKernelInstrumentor

trace_provider = register(project_type=ProjectType.OBSERVE, project_name="my-sk-app")

# Enables native diagnostics in process. Does not wrap Kernel.invoke.
SemanticKernelInstrumentor().instrument(tracer_provider=trace_provider, sensitive=False)

# ... build your Kernel / ChatCompletionAgent as usual ...
with using_session("conversation-123"):   # Semantic Kernel emits no thread id; set the session yourself
    response = await agent.get_response("What is the weather in Paris?")

trace_provider.force_flush()   # short scripts: the batch exporter sends in the background

instrument() is process-wide and idempotent: calling it twice, or on two instances, installs one processor. uninstrument() restores Semantic Kernel's switches and tracers and removes the processor. If you also put a SemanticKernelSpanProcessor on the provider yourself, instrument() does not take it over: it adds its own (with a warning), and uninstrument() removes and shuts down only that one. Use one or the other.

A runnable example with a mocked model (no model key) is in examples/basic_agent.py.

Ordering and other instrumentors

• Call instrument() after any add_span_processor() of your own on the provider register() returned. fi_instrumentation's TracerProvider.add_span_processor treats the processor chain register() set up as a replaceable default: its first call shuts down and empties the whole chain before adding yours (fi_instrumentation/otel.py, TracerProvider.add_span_processor; existing fi_instrumentation behaviour). That also removes this package's processor, while instrument() still reports instrumented and Semantic Kernel keeps emitting, so spans export without span kinds, session copy or TraceConfig hiding. If you must add a processor later, call uninstrument() and then instrument() again (covered by test_later_add_span_processor_drops_ours_until_reinstrumented).
• Do not also enable a client-level instrumentor for the same model calls (traceAI-openai, traceAI-anthropic, traceAI-litellm, ...). Semantic Kernel already puts gen_ai.usage.input_tokens / gen_ai.usage.output_tokens on its chat span, and this package keeps them there. A client instrumentor adds a second LLM span for the same request with the same tokens, and the processor only touches Semantic Kernel spans, so Observe's trace and session token sums count every model call twice. Use one or the other: this package for the Semantic Kernel tree, or the client instrumentor alone.

Sensitive content

Warning. sensitive=True turns on Semantic Kernel's sensitive diagnostics. Semantic Kernel then puts agent input and output messages, tool-call arguments and tool results on spans, this package adds model-call prompts and completions to the chat spans, and all of it is copied to input.value / output.value. Leave sensitive=False (the default) unless you want that text stored in Future AGI.

• With sensitive=False, instrument() forces Semantic Kernel's sensitive switch off even if your environment or .env turned it on (a warning is logged), and the processor removes gen_ai.input.messages, gen_ai.output.messages, gen_ai.tool.call.arguments, gen_ai.tool.call.result, input.value and output.value from Semantic Kernel spans.
• With sensitive=True, TraceConfig and its environment variables still apply. instrument(config=TraceConfig(hide_inputs=True)) or FI_HIDE_INPUTS=true drops tool arguments, agent and model-call input messages, input.value and input.mime_type; hide_outputs / FI_HIDE_OUTPUTS=true drops tool results, agent and model-call output messages, output.value and output.mime_type. The keys are removed, not replaced with __REDACTED__. Without config, a TraceConfig() is built from the environment when instrument() runs. Other TraceConfig fields (hide_input_messages, hide_output_messages, hide_input_text, hide_output_text, image and embedding limits, pii_redaction, hide_llm_invocation_parameters) are not applied by this package. A config that is not a TraceConfig raises TypeError.
• Model-call (chat) messages: Semantic Kernel writes them only to Python logging (model_diagnostics/decorators.py _set_completion_input / _set_completion_response), not to the span. With sensitive=True, instrument() puts a proxy in front of that module's logger that also appends each message to the current chat span as gen_ai.input.messages / gen_ai.output.messages (JSON lists of Semantic Kernel's message dicts), and uninstrument() puts the original back. The proxy passes every call on to the real logger unchanged, so the records still reach whatever handlers your Python logging has at INFO. With sensitive=False no proxy is installed.
• Errors: Semantic Kernel records the exception on the span (type, message and a stack trace with local file paths). The processor always removes exception.stacktrace. With sensitive=False it also removes exception.message and cuts the ERROR status description down to the exception type (RuntimeError), because messages often echo tool arguments or model output. exception.type, error.type and the ERROR status stay. With sensitive=True the message is kept.

Spans and attributes

Spans Semantic Kernel 1.44.1 emits once diagnostics are on, and what the processor adds:

Typical tree for an agent that calls one tool:

invoke_agent Assistant                    AGENT
└─ AutoFunctionInvocationLoop             CHAIN
   ├─ chat gpt-4o-mini                    LLM   (finish_reason tool_calls)
   ├─ execute_tool Weather-get_weather    TOOL
   └─ chat gpt-4o-mini                    LLM   (finish_reason stop)

Not available: retrieval spans (memory/vector connectors were not inspected), cache and reasoning token counts, cost, a thread or conversation id from Semantic Kernel itself.

Failure behavior

• Collector down or slow: the OTLP exporter logs and drops; your kernel call returns normally (covered by test_collector_down_never_breaks_the_kernel).
• The processor swallows its own exceptions and never raises into the OpenTelemetry SDK.
• instrument() does not raise into your startup when a Semantic Kernel module it expects cannot be imported (the diagnostics are experimental upstream and may move in a later release). It logs a WARNING naming the module, skips it, and instruments the rest. A provider that is not an OpenTelemetry SDK TracerProvider (for example the global ProxyTracerProvider before any provider is set) gets a WARNING and nothing is changed. uninstrument() is safe in both cases.
• A full batch queue (default 2048) drops spans inside OpenTelemetry; the agent does not see it.
• If a stream is abandoned mid-way, Semantic Kernel owns the span end; this package does not open a second span to close it.

Tested versions

Semantic Kernel's own classifiers stop at Python 3.12; both versions install on 3.13 and the full suite passes there.

Semantic Kernel added agent diagnostics in 1.19.0, but the surface this package reads (three module-level settings objects, chat / invoke_agent / execute_tool operation strings, gen_ai.input.messages agent keys) is identical from 1.38.0 to 1.44.1 and differs before it (for example chat.completions and gen_ai.agent.invocation_input up to 1.37.x). Hence the 1.38.0 floor.

Azure AI Inference connector: it builds its own ModelDiagnosticSettings() from the environment at call time (connectors/ai/azure_ai_inference/services/azure_ai_inference_tracing.py 25) and is not switched by instrument().

Tests

PYTHONPATH="python/frameworks/semantic-kernel:python:python/tests" uv run --no-project --python 3.11 --prerelease=allow \
  --with pytest --with opentelemetry-api --with opentelemetry-sdk --with opentelemetry-exporter-otlp-proto-http \
  --with wrapt --with requests --with jsonschema --with protobuf --with opentelemetry-proto \
  --with 'semantic-kernel==1.44.1' \
  pytest python/frameworks/semantic-kernel/tests -q -p no:cacheprovider --noconftest -o addopts=''

Swap --python for 3.10, 3.12 or 3.13 and semantic-kernel==1.38.0 for the floor. For the OpenTelemetry floor, pin opentelemetry-api==1.29.0, opentelemetry-sdk==1.29.0, opentelemetry-exporter-otlp-proto-http==1.29.0 and opentelemetry-proto==1.29.0 in the --with list.

semantic-kernel 1.38.0+ depends on the pre-release azure-ai-agents>=1.2.0b3, so uv needs --prerelease=allow (or an explicit azure-ai-agents>=1.2.0b3 pin). wrapt is there for fi_instrumentation, not for this package.

• tests/test_processor.py: synthetic spans with Semantic Kernel's exact names and keys, including an id-less tool call under the auto function invocation loop and the TraceConfig hide flags.
• tests/test_instrument.py: switches flip without env vars, Kernel.invoke / KernelFunction.invoke are untouched, one processor after two calls, tracers routed and restored, a real id-less auto-invoked tool, warn-and-skip for a moved module or a non-SDK provider, a user-installed processor left alone, TraceConfig / FI_HIDE_* on real spans, and the add_span_processor ordering note.
• tests/test_contract_harness.py: real Semantic Kernel, real register() exporter, shared harness Receiver, loopback fake OpenAI server, placeholder keys.