LLM Tracekit

Open-source observability for your LLM application, based on OpenTelemetry.

LLM Tracekit is a set of OpenTelemetry instrumentations that gives you complete observability over your LLM application. Because it uses OpenTelemetry under the hood, it can be connected to your existing observability solutions - Coralogix, Datadog, Honeycomb, and others.

🚀 Getting Started

Install the instrumentation for your LLM provider:

pip install llm-tracekit-openai             # For OpenAI
pip install llm-tracekit-bedrock            # For AWS Bedrock
pip install llm-tracekit-gemini             # For Google Gemini
pip install llm-tracekit-google-adk         # For Google ADK
pip install llm-tracekit-litellm            # For LiteLLM
pip install llm-tracekit-langchain          # For LangChain
pip install llm-tracekit-langgraph          # For LangGraph
pip install llm-tracekit-openai-agents      # For OpenAI Agents SDK
pip install llm-tracekit-strands            # For Strands Agents
pip install llm-tracekit-anthropic          # For Anthropic (Claude API)
pip install llm-tracekit-microsoft-foundry  # For Microsoft Foundry

Then instrument your code:

from llm_tracekit.openai import OpenAIInstrumentor, setup_export_to_coralogix

setup_export_to_coralogix(
    service_name="my-ai-service",
    capture_content=True,
)

OpenAIInstrumentor().instrument()

from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}],
)

🪗 What do we instrument?

LLM Providers

Frameworks

📖 Usage

Setting up tracing

Export to Coralogix

from llm_tracekit.openai import setup_export_to_coralogix

setup_export_to_coralogix(
    service_name="ai-service",
    application_name="ai-application",
    subsystem_name="ai-subsystem",
    capture_content=True,
)

🛡️ Guardrails

LLM Tracekit also includes Coralogix Guardrails - a client for protecting your LLM applications with content moderation, PII detection, prompt injection detection, and more.

See the Guardrails documentation for details.

📚 Documentation

For detailed documentation on each instrumentation, see the individual READMEs:

• OpenAI
• AWS Bedrock
• Google Gemini
• Google ADK
• LiteLLM
• LangChain
• LangGraph
• OpenAI Agents SDK
• Strands Agents
• Anthropic
• Microsoft Foundry

📜 License

Apache 2.0 - See LICENSE for details.