Token Usage

Chat and prompt completions record detailed usage when Portkey returns it, for synchronous, asynchronous, and streaming calls:

• usage.prompt_tokens_details.cache_write_tokens records gen_ai.usage.cache_write.input_tokens, falling back to usage.cache_creation_input_tokens when the nested count is unavailable.

• usage.prompt_tokens_details.cached_tokens records gen_ai.usage.cache_read.input_tokens, falling back to usage.cache_read_input_tokens when the nested count is unavailable. The two representations are not added together.

• Explicit text_tokens, image_tokens, and audio_tokens under usage.prompt_tokens_details record the corresponding gen_ai.usage.{text,image,audio}.input_tokens attributes.

• Explicit text_tokens, image_tokens, and audio_tokens under usage.completion_tokens_details record gen_ai.usage.{text,image,audio}.output_tokens.

• usage.completion_tokens_details.reasoning_tokens records gen_ai.usage.reasoning.output_tokens without adding to the reported output-token total.

Availability depends on the provider and model. Aggregate input and output totals are preserved, and modality counts are never inferred from those totals. No per-modality cache counts are inferred from the aggregate cache-read count. Only non-negative integer counts, excluding booleans, are accepted. Missing or invalid counts are omitted; streaming updates retain previously reported counts when later chunks omit them. These attributes do not require message-content capture.

The fallback cache fields follow Portkey’s prompt caching response format; nested details include fields preserved by its OpenAI-compatible response passthrough.

Capture Message Content

By default, prompts and completions are not captured. To capture message content, set the environment variable OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT to one of no_content, span_only, event_only, or span_and_event:

export OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=span_and_event

Uploading Prompts and Completions

To enable the built-in upload completion hook, set:

• OTEL_INSTRUMENTATION_GENAI_COMPLETION_HOOK=upload

• OTEL_INSTRUMENTATION_GENAI_UPLOAD_BASE_PATH to an fsspec-compatible URI/path (e.g. /path/to/prompts or gs://my_bucket).

Install the upload extra to pull in fsspec:

pip install opentelemetry-util-genai[upload]

You can also programmatically pass a custom hook when calling instrument():

PortkeyInstrumentor().instrument(completion_hook=my_custom_hook)

Uninstrument

To remove instrumentation from Portkey AI clients, call uninstrument():

PortkeyInstrumentor().uninstrument()