openlit 1.22.4

OpenTelemetry-native Auto instrumentation library for monitoring LLM Applications and GPUs, facilitating the integration of observability into your GenAI-driven projects

OpenTelemetry Auto-Instrumentation for GenAI & LLM Applications

Documentation | Quickstart | Python SDK

OpenLIT Python SDK is an OpenTelemetry-native Auto instrumentation library for monitoring LLM Applications, facilitating the integration of observability into your GenAI-driven projects. Designed with simplicity and efficiency, OpenLIT offers the ability to embed observability into your GenAI-driven projects effortlessly using just a single line of code.

Whether you're directly using LLM Libraries like OpenAI, Anthropic or building complex RAG Agents using Langchain, OpenLIT seamlessly integrates observability into your applications, ensuring enhanced performance and reliability across diverse scenarios.

This project adheres to the Semantic Conventions proposed by the OpenTelemetry community. You can check out the current definitions here.

Auto Instrumentation Capabilities

LLMs	Vector DBs	Frameworks	GPUs
✅ OpenAI	✅ ChromaDB	✅ Langchain	✅ NVIDIA GPUs
✅ Ollama	✅ Pinecone	✅ LiteLLM
✅ Anthropic	✅ Qdrant	✅ LlamaIndex
✅ GPT4All	✅ Milvus	✅ Haystack
✅ Cohere		✅ EmbedChain
✅ Mistral		✅ Guardrails
✅ Azure OpenAI
✅ Azure AI Inference
✅ GitHub AI Models
✅ HuggingFace Transformers
✅ Amazon Bedrock
✅ Vertex AI
✅ Groq
✅ ElevenLabs
✅ vLLM
✅ OLA Krutrim
✅ Google AI Studio
✅ NVIDIA NIM

Supported Destinations

• ✅ OpenTelemetry Collector
• ✅ Prometheus + Tempo
• ✅ Prometheus + Jaeger
• ✅ Grafana Cloud
• ✅ New Relic
• ✅ Elastic
• ✅ HyperDX
• ✅ DataDog
• ✅ SigNoz
• ✅ OneUptime
• ✅ Dynatrace
• ✅ OpenObserve
• ✅ Highlight.io

💿 Installation

pip install openlit

🚀 Getting Started

Step 1: Install OpenLIT

Open your command line or terminal and run:

pip install openlit

Step 2: Initialize OpenLIT in your Application

Integrating the OpenLIT into LLM applications is straightforward. Start monitoring for your LLM Application with just two lines of code:

import openlit

openlit.init()

To forward telemetry data to an HTTP OTLP endpoint, such as the OpenTelemetry Collector, set the otlp_endpoint parameter with the desired endpoint. Alternatively, you can configure the endpoint by setting the OTEL_EXPORTER_OTLP_ENDPOINT environment variable as recommended in the OpenTelemetry documentation.

💡 Info: If you dont provide otlp_endpoint function argument or set the OTEL_EXPORTER_OTLP_ENDPOINT environment variable, OpenLIT directs the trace directly to your console, which can be useful during development.

To send telemetry to OpenTelemetry backends requiring authentication, set the otlp_headers parameter with its desired value. Alternatively, you can configure the endpoint by setting the OTEL_EXPORTER_OTLP_HEADERS environment variable as recommended in the OpenTelemetry documentation.

Example

---

Initialize using Function Arguments

---

Add the following two lines to your application code:

import openlit

openlit.init(
  otlp_endpoint="YOUR_OTEL_ENDPOINT", 
  otlp_headers ="YOUR_OTEL_ENDPOINT_AUTH"
)

---

Initialize using Environment Variables

---

Add the following two lines to your application code:

import openlit

openlit.init()

Then, configure the your OTLP endpoint using environment variable:

export OTEL_EXPORTER_OTLP_ENDPOINT = "YOUR_OTEL_ENDPOINT"
export OTEL_EXPORTER_OTLP_HEADERS = "YOUR_OTEL_ENDPOINT_AUTH"

---

Step 3: Visualize and Optimize!

With the LLM Observability data now being collected and sent to OpenLIT, the next step is to visualize and analyze this data to get insights into your LLM application’s performance, behavior, and identify areas of improvement.

To begin exploring your LLM Application's performance data within the OpenLIT UI, please see the Quickstart Guide.

If you want to integrate and send metrics and traces to your existing observability tools, refer to our Connections Guide for detailed instructions.

Configuration

Below is a detailed overview of the configuration options available, allowing you to adjust OpenLIT's behavior and functionality to align with your specific observability needs:

Argument	Description	Default Value	Required
environment	The deployment environment of the application.	"default"	Yes
application_name	Identifies the name of your application.	"default"	Yes
tracer	An instance of OpenTelemetry Tracer for tracing operations.	None	No
meter	An OpenTelemetry Metrics instance for capturing metrics.	None	No
otlp_endpoint	Specifies the OTLP endpoint for transmitting telemetry data.	None	No
otlp_headers	Defines headers for the OTLP exporter, useful for backends requiring authentication.	None	No
disable_batch	A flag to disable batch span processing, favoring immediate dispatch.	False	No
trace_content	Enables tracing of content for deeper insights.	True	No
disabled_instrumentors	List of instrumentors to disable.	None	No
disable_metrics	If set, disables the collection of metrics.	False	No
pricing_json	URL or file path of the pricing JSON file.	https://github.com/openlit/openlit/blob/main/assets/pricing.json	No
collect_gpu_stats	Flag to enable or disable GPU metrics collection.	False	No

🌱 Contributing

Whether it's big or small, we love contributions 💚. Check out our Contribution guide to get started

Unsure where to start? Here are a few ways to get involved:

• Join our Slack or Discord community to discuss ideas, share feedback, and connect with both our team and the wider OpenLIT community.

Your input helps us grow and improve, and we're here to support you every step of the way.

💚 Community & Support

Connect with the OpenLIT community and maintainers for support, discussions, and updates:

• 🌟 If you like it, Leave a star on our GitHub
• 🌍 Join our Slack or Discord community for live interactions and questions.
• 🐞 Report bugs on our GitHub Issues to help us improve OpenLIT.
• 𝕏 Follow us on X for the latest updates and news.