llumo-inference 0.1.0

Llumo Telemetry SDK for LLM Observability

LLumo Telemetry SDK (Python)

A powerful telemetry SDK designed to instrument LLM operations via OpenAI, Anthropic, and LangChain and send formatted telemetry data to your backend telemetry server.

Installation

1. Create a virtual environment:

   python -m venv .venv
   source .venv/bin/activate  # On Windows: .venv\Scripts\activate
   

2. Install dependencies:

   pip install -r requirements.txt
   

Setup Guide

Place this initialization setup at the entry point of your application, before you initialize any LLM clients.

```python
from llumo_inference import initSDK, TelemetryConfig

# Initialize the telemetry
config = TelemetryConfig(
    endpoint='http://localhost:4455/api/v1/telemetry',  # Your custom telemetry API endpoint
    authToken='your-auth-token',  # Optional Auth Bearer Token
    flushDelayMillis=500  # Span buffer flush interval (def: 500ms)
)

# Pass optional library instances if you need manual instrumentation
# config.libraries = {
#     "OpenAI": openai_client,
#     "Anthropic": anthropic_client
# }

initSDK(config)

print("Telemetry configured successfully.")

Configuration Options

Option	Type	Required	Description
endpoint	string	Yes	The URL of your telemetry ingestion server
authToken	string	No	Optional Bearer token inside Auth header
flushDelayMillis	int	No	Interval to ship logs in milliseconds. Defaults to 500ms
maxExportBatchSize	int	No	Max payload size limits. Defaults to 50
libraries	dict	No	Optional dict for injecting specific AI client instances

Features

• Built-in Instrumentations: Supports OpenAI, Anthropic, Gemini (Vertex AI & Google GenAI), LangChain, requests, and urllib3.
• Auto Data Sanitation: MongoDB-compliant key formatting automatically escapes problematic fields (. and $) before transmission.
• Trace Exporters: Uses BatchSpanProcessor with a custom FormattingExporter for structured, ready-to-consume payloads.
• Performance: Asynchronous-style exporting via OTel's native batching to minimize impact on application latency.