loongsuite-instrumentation-litellm 0.4.0

LoongSuite LiteLLM instrumentation

This library provides automatic instrumentation for the LiteLLM library, which provides a unified interface to 100+ LLM providers.

Installation

git clone https://github.com/alibaba/loongsuite-python-agent.git
cd loongsuite-python-agent
pip install ./instrumentation-loongsuite/loongsuite-instrumentation-litellm

Configuration

The instrumentation can be enabled/disabled using environment variables:

• ENABLE_LITELLM_INSTRUMENTOR: Enable/disable instrumentation (default: true)

Usage

from opentelemetry.instrumentation.litellm import LiteLLMInstrumentor
import litellm

# Instrument LiteLLM
LiteLLMInstrumentor().instrument()

# Use LiteLLM as normal
response = litellm.completion(
    model="gpt-3.5-turbo",
    messages=[{"role": "user", "content": "Hello!"}]
)

Features

This instrumentation automatically captures:

• LLM completion calls (sync and async)

• Streaming completions

• Embedding calls

• Retry mechanisms

• Tool/function calls

• Request and response metadata

• Token usage

• Model information

The instrumentation follows OpenTelemetry semantic conventions for GenAI operations.

References

• OpenTelemetry LiteLLM Instrumentation

• OpenTelemetry Project

• LiteLLM Documentation