atla-insights 0.0.35

Atla is a platform for monitoring and improving AI agents.

Atla Insights

Atla Insights is a platform for monitoring and improving AI agents.

Getting started

To get started with Atla Insights, you can either follow the instructions below, or let an agent instrument your code for you.

• If you are using Claude Code: copy the contents of Claude.md.
• If you are not using Claude Code: copy the contents of onboarding.txt and paste them into your AI agent of choice.

Installation

pip install atla-insights

To install package-specific dependencies:

pip install "atla-insights[litellm]"

Usage

Configuration

Before using Atla Insights, you need to configure it with your authentication token:

from atla_insights import configure

# Run this command at the start of your application.
configure(token="<MY_ATLA_INSIGHTS_TOKEN>")

You can retrieve your authentication token from the Atla Insights platform.

Environment Configuration

Separate traces between development and production environments:

import os

from atla_insights import configure

# Development environment
configure(token="<TOKEN>", environment="dev")

# Production environment (default)
configure(token="<TOKEN>", environment="prod")

# Via environment variable
os.environ["ATLA_INSIGHTS_ENVIRONMENT"] = "dev"
configure(token="<TOKEN>")  # Uses "dev" from env var

Instrumentation

In order for spans/traces to become available in your Atla Insights dashboard, you will need to add some form of instrumentation.

As a starting point, you will want to instrument your GenAI library of choice.

See the section below to find out which frameworks & providers we currently support.

All instrumentation methods share a common interface, which allows you to do the following:

• Session-wide (un)instrumentation: You can manually enable/disable instrumentation throughout your application.

from atla_insights import configure, instrument_my_framework, uninstrument_my_framework

configure(...)
instrument_my_framework()

# All framework code from this point onwards will be instrumented

uninstrument_my_framework()

# All framework code from this point onwards will **no longer** be instrumented

• Instrumented contexts: All instrumentation methods also behave as context managers that automatically handle (un)instrumentation.

from atla_insights import configure, instrument_my_framework

configure()

with instrument_my_framework():
    # All framework code inside the context will be instrumented

# All framework code outside the context **not** be instrumented

Instrumentation Support

Providers

We currently support the following LLM providers:

Provider	Instrumentation Function	Notes
Anthropic	instrument_anthropic	Also supports AnthropicBedrock client from Anthropic
Google GenAI	instrument_google_genai	E.g., Gemini
LiteLLM	instrument_litellm	Supports all available models in the LiteLLM framework
OpenAI	instrument_openai	Includes Azure OpenAI
Bedrock	instrument_bedrock

⚠️ Note that, by default, instrumented LLM calls will be treated independently from one another. In order to logically group LLM calls into a trace, you will need to group them as follows:

from atla_insights import configure, instrument, instrument_litellm
from litellm import completion

configure(...)
instrument_litellm()

# The LiteLLM calls below will belong to **separate traces**
result_1 = completion(...)
result_2 = completion(...)

@instrument("My agent doing its thing")
def run_my_agent() -> None:
    # The LiteLLM calls within this function will belong to the **same trace**
    result_1 = completion(...)
    result_2 = completion(...)
    ...

Frameworks

We currently support the following frameworks:

Framework	Instrumentation Function	Notes
Agno	instrument_agno	Supported with openai, google-genai, litellm and/or anthropic models*
BAML	instrument_baml	Supported with openai, anthropic or bedrock models*
Claude Agent SDK	instrument_claude_agent_sdk
Claude Code SDK	instrument_claude_code_sdk
CrewAI	instrument_crewai
Google ADK	instrument_google_adk
LangChain	instrument_langchain	This includes e.g., LangGraph as well
MCP	instrument_mcp	Only includes context propagation. You will need to instrument the model calling the MCP server separately.
OpenAI Agents	instrument_openai_agents	Supported with openai, google-genai, litellm and/or anthropic models*
Pydantic AI	instrument_pydantic_ai
Smolagents	instrument_smolagents	Supported with openai, google-genai, litellm and/or anthropic models*

⚠️ *Note that some frameworks do not provide their own LLM interface. In these cases, you will need to instrument both the framework and the underlying LLM provider(s) as follows:

from atla_insights import configure, instrument, instrument_agno

configure(...)

# If you are using a single LLM provider (e.g., via `OpenAIChat`).
instrument_agno("openai")

# If you are using multiple LLM providers (e.g., `OpenAIChat` and `Claude`).
instrument_agno(["anthropic", "openai"])

Manual instrumentation

It is also possible to manually record LLM generations using the lower-level span SDK.

from atla_insights.span import start_as_current_span

with start_as_current_span("my-llm-generation") as span:
    # Run my LLM generation via an unsupported framework.
    input_messages = [{"role": "user", "content": "What is the capital of France?"}]
    tools = [
        {
            "type": "function",
            "function": {
                "name": "get_capital",
                "parameters": {"type": "object", "properties": {"country": {"type": "string"}}},
            },
        }
    ]
    result = my_client.chat.completions.create(messages=input_messages, tools=tools)

    # Manually record LLM generation.
    span.record_generation(
        input_messages=input_messages,
        output_messages=[choice.message for choice in result.choices],
        tools=tools,
    )

Note that the expected data format are OpenAI Chat Completions compatible messages / tools.

Git integration

In order to make use of advanced Atla features, you can use the SDK to connect to git.

If your deployment has access to the repo's .git file and has the git CLI installed, this information will get collected automatically. You can opt out by setting ATLA_DISABLE_GIT_TRACKING=1 in your environment.

If your deployment does not have access to the repo's .git file or does not have the git CLI installed, you can manually specify the necessary git info via environment variables:

Environment variable	Git command	Example value
ATLA_GIT_REPO	git remote get-url origin	https://github.com/atla-ai/atla-insights-sdk
ATLA_GIT_BRANCH	git symbolic-ref --short HEAD	feature/my-new-experiment
ATLA_GIT_COMMIT_HASH	git rev-parse HEAD	269d6bfa34af668309df396ee5bce7e54257c320
ATLA_GIT_COMMIT_MESSAGE	git log -1 --pretty=%B	Implement my new feature
ATLA_GIT_COMMIT_TIMESTAMP	git remote get-url origin	2025-10-01T18:45:47+01:00
ATLA_GIT_SEMVER	git describe --tags --abbrev=0	v1.7

Adding metadata

You can attach metadata to a run that provides additional information about the specs of that specific workflow. This can include various system settings, prompt versions, etc.

from atla_insights import configure

# We can define some system settings, prompt versions, etc. we'd like to keep track of.
metadata = {
    "environment": "dev",
    "prompt-version": "v1.4",
    "model": "gpt-4o-2024-08-06",
    "run-id": "my-test",
}

# Any subsequent generated traces will inherit the metadata specified here.
configure(
    token="<MY_ATLA_INSIGHTS_TOKEN>",
    metadata=metadata,
)

Dynamic metadata

Metadata set with the configure function will be attached to all traces. You can also set metadata dynamically during runtime. This is useful, for example, to "tag" specific traces with information that is only available during runtime.

from atla_insights import set_metadata

# ... within instrumented context ...
set_metadata({"some_key": "some_value", "other_key": "other_value"})

Tool invocations

If you want to ensure your function-based tool calls are logged correctly, you can wrap them using the @tool decorator as follows:

from atla_insights import tool

@tool
def my_tool(my_arg: str) -> str:
    return "some-output"

⚠️ Note that if you are using an instrumented framework, you do not need to manually decorate your tools in this way.

Sampling

By default, Atla Insights will instrument & log all traces. In high-throughput scenarios, you may not want to log every trace you produce. In these cases, you can specify a sampler at configuration time.

• Setting a trace ratio:

  If you want to sample a fixed ratio of traffic, we recommend using a TraceRatioSampler. This is the simplest & most computationally efficient way to sample traces.

  from atla_insights import configure
  from atla_insights.sampling import TraceRatioSampler
  
  configure(
      token="<MY_ATLA_INSIGHTS_TOKEN>",
      sampler=TraceRatioSampler(rate=0.10),  # logging 10% of traces
  )
  

• Sampling decision based on metadata:

  If you want more flexibility, you can define a custom sampling decision function based on metadata.

  import random
  from typing import Optional
  
  from atla_insights import configure
  from atla_insights.sampling import MetadataSampler
  
  def sampling_fn(metadata: Optional[dict[str, str]]) -> bool:
      """Custom sampling decision function.
  
      :param metadata (Optional[dict[str, str]]): The metadata to sample.
      :return (bool): Whether to sample the trace.
      """
      if metadata is None:
          return False
  
      if metadata.get("feature") == "feature_1":
          # Sample 50% of traffic for traces tagged as feature 1
          return bool(random.random() < 0.50)
  
      # Sample 10% of traffic otherwise
      return bool(random.random() < 0.10)
  
  
  configure(
      token="<MY_ATLA_INSIGHTS_TOKEN>",
      sampler=MetadataSampler(sampling_fn),
  )
  

  Note that this is a more computationally intensive sampling method as we need to keep all spans in a trace alive in-memory until the entire trace ends. As metadata is mutable, we can only check the sampling decision function at the end of each trace.

• Custom sampling method: If you want to implement your own custom sampling method, you can pass in your own OpenTelemery Sampler.

  import os
  from atla_insights import configure
  from opentelemetry.sdk.trace.sampling import Sampler
  
  class MySampler(Sampler):
      ...
  
  configure(
      token=os.environ["ATLA_INSIGHTS_TOKEN"],
      sampler=MySampler(),
  )
  

  ⚠️ Note that the Atla Insights platform is not intended to work well with partial traces. Therefore, we highly recommend using either ParentBased or StaticSampler samplers. This ensures either all traces are treated the same way or all spans in the same trace are treated the same way.

  from atla_insights import configure
  from opentelemetry.sdk.trace.sampling import ParentBased, Sampler
  
  class MySampler(Sampler):
      ...
  
  configure(
      token=os.environ["ATLA_INSIGHTS_TOKEN"],
      sampler=ParentBased(root=MySampler()),
  )
  

Adding custom metrics

You can add custom evaluation metrics to your trace.

from atla_insights import instrument, set_custom_metrics

@instrument()
def my_function():
    # Some GenAI logic here
    eval_result = False
    set_custom_metrics({"my_metric": {"data_type": "boolean", "value": eval_result}})

The permitted data_type fields are:

• likert_1_to_5: a numeric 1-5 scale. In this case, value is expected to be an int between 1 and 5.
• boolean: a boolean scale. In this case, value is expected to be a bool.

The primary intended use case is logging custom code evals that benefit from being in the active runtime environment. You can, however, log any arbitrary metric - including custom LLMJ eval results.

Marking trace success / failure

The logical notion of success or failure plays a prominent role in the observability of (agentic) GenAI applications.

Therefore, the atla_insights package offers the functionality to mark a trace as a success or a failure like follows:

from atla_insights import (
    configure,
    instrument,
    instrument_openai,
    mark_failure,
    mark_success,
)
from openai import OpenAI

configure(...)
instrument_openai()

client = OpenAI()

@instrument("My agent doing its thing")
def run_my_agent() -> None:
    result = client.chat.completions.create(
        model=...,
        messages=[
            {
                "role": "user",
                "content": "What is 1 + 2? Reply with only the answer, nothing else.",
            }
        ]
    )
    response = result.choices[0].message.content

    # Note that you could have any arbitrary success condition, including LLMJ-based evaluations
    if response == "3":
        mark_success()
    else:
        mark_failure()

⚠️ Note that you should use this marking functionality within an instrumented function.

Running experiments

You can run experiments by using the run_experiment context manager.

from atla_insights import run_experiment

with run_experiment(
    experiment_name="my-experiment-name",  # optional: defaults to random identifier
    description="Testing out shiny new feature",  # optional
):
    # Your experiment code here

These experiments will show up in your Atla Insights workbench.

Compatibility with existing observability

As atla_insights provides its own instrumentation, we should note potential interactions with our instrumentation / observability providers.

atla_insights instrumentation is generally compatible with most popular observability platforms.

E.g., the following code snippet will make tracing available in both Atla and Langfuse.

from atla_insights import configure, instrument_openai
from langfuse.openai import OpenAI

configure(...)

instrument_openai()

client = OpenAI()
client.chat.completions.create(...)

OpenTelemetry compatibility

The Atla Insights SDK is built on the OpenTelemetry standard and fully compatible with other OpenTelemetry services.

If you have an existing OpenTelemetry setup (e.g., by setting the relevant otel environment variables), Atla Insights will be additive to this setup. I.e., it will add additional logging on top of what is already getting logged.

If you do not have an existing OpenTelemetry setup, Atla Insights will initialize a new (global) tracer provider.

Next to the above, you also have the ability to add any arbitrary additional span processors by following the example below:

from atla_insights import configure
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace.export import SimpleSpanProcessor

# This is the otel traces endpoint for my provider of choice.
my_otel_endpoint = "https://my-otel-provider/v1/traces"

my_span_exporter = OTLPSpanExporter(endpoint=my_otel_endpoint)
my_span_processor = SimpleSpanProcessor(my_span_exporter)

configure(
    token="<MY_ATLA_INSIGHTS_TOKEN>",
    # This will ensure traces get sent to my otel provider of choice
    additional_span_processors=[my_span_processor],
)

More examples

More specific examples can be found in the examples/ folder.

Data API

The Atla Insights SDK includes a data API client for programmatically accessing your traces and analytics data.

Usage

from atla_insights.client import Client

# Initialize the client with your API key
client = Client(api_key="your_api_key_here")

# List traces with optional filters
traces = client.list_traces(
    page_size=50,
    metadata_filter=[{"key": "environment", "value": "prod"}]
)

# Get detailed trace information
trace_details = client.get_trace("trace_id_123")

# Bulk retrieve multiple traces
traces = client.get_traces(["trace_1", "trace_2", "trace_3"])

Available Methods

• list_traces() - Retrieve paginated list of traces with optional filtering
• get_trace(trace_id) - Get detailed information for a specific trace
• get_traces(trace_ids) - Bulk retrieve multiple traces by ID

The client returns structured data objects with full type hints for easy integration into your workflows.

See the examples/ directory for additional usage examples and integration patterns.