Skip to main content

OpenTelemetry instrumentation for LiteLLM

Project description

Litellm OpenTelemetry Integration

Overview

This integration provides support for using OpenTelemetry with the Litellm framework. It enables tracing and monitoring of applications built with Litellm.

Installation

  1. Install traceAI Litellm
pip install pip install traceAI-litellm

Set Environment Variables

Set up your environment variables to authenticate with FutureAGI

import os

os.environ["FI_API_KEY"] = FI_API_KEY
os.environ["FI_SECRET_KEY"] = FI_SECRET_KEY

Quickstart

Register Tracer Provider

Set up the trace provider to establish the observability pipeline. The trace provider:

from fi_instrumentation import register
from fi_instrumentation.fi_types import ProjectType

trace_provider = register(
    project_type=ProjectType.OBSERVE,
    project_name="litellm_app"
)

Configure Litellm Instrumentation

Instrument the Litellm client to enable telemetry collection. This step ensures that all interactions with the Litellm SDK are tracked and monitored.

from traceai_litellm import LiteLLMInstrumentor

LiteLLMInstrumentor().instrument(tracer_provider=trace_provider)

Create Litellm Components

Set up your Litellm client with built-in observability.

import asyncio
import litellm

async def run_examples():
    # Simple single message completion call
    litellm.completion(
        model="gpt-3.5-turbo",
        messages=[{"content": "What's the capital of China?", "role": "user"}],
    )

    # Multiple message conversation completion call with added param
    litellm.completion(
        model="gpt-3.5-turbo",
        messages=[
            {"content": "Hello, I want to bake a cake", "role": "user"},
            {
                "content": "Hello, I can pull up some recipes for cakes.",
                "role": "assistant",
            },
            {"content": "No actually I want to make a pie", "role": "user"},
        ],
        temperature=0.7,
    )

    # Multiple message conversation acompletion call with added params
    await litellm.acompletion(
        model="gpt-3.5-turbo",
        messages=[
            {"content": "Hello, I want to bake a cake", "role": "user"},
            {
                "content": "Hello, I can pull up some recipes for cakes.",
                "role": "assistant",
            },
            {"content": "No actually I want to make a pie", "role": "user"},
        ],
        temperature=0.7,
        max_tokens=20,
    )

    # Completion with retries
    litellm.completion_with_retries(
        model="gpt-3.5-turbo",
        messages=[{"content": "What's the highest grossing film ever", "role": "user"}],
    )

    # Embedding call
    litellm.embedding(
        model="text-embedding-ada-002", input=["good morning from litellm"]
    )

    # Asynchronous embedding call
    await litellm.aembedding(
        model="text-embedding-ada-002", input=["good morning from litellm"]
    )

    # Image generation call
    litellm.image_generation(model="dall-e-2", prompt="cute baby otter")

    # Asynchronous image generation call
    await litellm.aimage_generation(model="dall-e-2", prompt="cute baby otter")

asyncio.run(run_examples())

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

traceai_litellm-0.1.8.tar.gz (7.3 kB view details)

Uploaded Source

Built Distribution

traceai_litellm-0.1.8-py3-none-any.whl (7.5 kB view details)

Uploaded Python 3

File details

Details for the file traceai_litellm-0.1.8.tar.gz.

File metadata

  • Download URL: traceai_litellm-0.1.8.tar.gz
  • Upload date:
  • Size: 7.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/2.0.0 CPython/3.13.0 Darwin/24.1.0

File hashes

Hashes for traceai_litellm-0.1.8.tar.gz
Algorithm Hash digest
SHA256 ccc58ded24e435c1aafac1403da8958ecbf2d75469b59c51c5133d9f9dd234bf
MD5 600a5f016d36f898f5bc3e0dd65bc8ea
BLAKE2b-256 56a0be57abbdab418738e54137cf43f32781bae557780648ba78060f4f825d2f

See more details on using hashes here.

File details

Details for the file traceai_litellm-0.1.8-py3-none-any.whl.

File metadata

  • Download URL: traceai_litellm-0.1.8-py3-none-any.whl
  • Upload date:
  • Size: 7.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/2.0.0 CPython/3.13.0 Darwin/24.1.0

File hashes

Hashes for traceai_litellm-0.1.8-py3-none-any.whl
Algorithm Hash digest
SHA256 15d1c7b762fae314ebd245b21c820c7fa2d0a0306ab9ec08035c4265eacc8afb
MD5 6c7ed7f7e1241d99a8a08c97934312c9
BLAKE2b-256 e8488a7f89e1a2580149a78f5fbc6461203816e4ca888d42348427939c44d5e9

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page