tapm-instrumentation-vllm 0.1b3

OpenTelemetry vLLM instrumentation

This library allows tracing and metrics collection for vLLM LLM serving framework.

Installation

pip install opentelemetry-instrumentation-vllm

Features

• Works with vLLM 0.7.x–0.10.x (set VLLM_USE_V1=1 to opt into the v1 engine when running 0.8.x–0.10.x).

• Captures spans for non-streaming and streaming create_completion and create_chat_completion calls, including prompt/response bodies, token usage and finish reasons.

• Hooks the internal LLM engine to emit spans with TTFT, queueing time, per-stage latency and sampling parameters.

• Hijacks vLLM’s Prometheus loggers so that all engine metrics are pushed through OpenTelemetry meters instead of relying on pull-based scraping.

Usage

from opentelemetry.instrumentation.vllm import VLLMInstrumentor

VLLMInstrumentor().instrument()

References

• OpenTelemetry vLLM Instrumentation

• OpenTelemetry Project

• vLLM