Blue Guardrails - Haystack

The Blue Guardrails Haystack integration instruments LLM and agent calls in Haystack and sends them as OpenTelemetry traces to the Blue Guardrails platform.

Use Blue Guardrails to monitor your agents and other GenAI applications in production or evaluate pre-deployment.

The Blue Guardrails reliability layer runs on ingested traces and detects issues like hallucinations, poor instruction-following, or tool-calling issues.

More info on Blue Guardrails in the official documentation.

Features

Install

Install the package with pip:

pip install blueguardrails-haystack

Or with uv:

uv add blueguardrails-haystack

The package supports Python 3.11 through 3.14.

Use

Trace an agent

To trace a Haystack agent, configure the Blue Guardrails tracer before you run the agent.

from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.tools import Tool

from blueguardrails_haystack import configure_blueguardrails_tracer


def get_weather(city: str) -> str:
    return f"The weather in {city} is sunny."


configure_blueguardrails_tracer(
    name="support-agent",
    tags={"environment": "development"},
)

weather_tool = Tool(
    name="get_weather",
    description="Returns the weather for a city.",
    parameters={
        "type": "object",
        "properties": {"city": {"type": "string"}},
        "required": ["city"],
    },
    function=get_weather,
)

agent = Agent(
    chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"),
    tools=[weather_tool],
    system_prompt="Use tools when they help answer the user.",
    exit_conditions=["text"],
    max_agent_steps=3,
)

result = agent.run(
    messages=[ChatMessage.from_user("What is the weather in Berlin?")]
)

print(result["last_message"].text)

Blue Guardrails receives a trace for each generator call the agent makes.

configure_blueguardrails_tracer() reads BLUE_GUARDRAILS_API_KEY by default. If you don't set BLUE_GUARDRAILS_API_KEY, pass api_key explicitly. It raises ValueError if neither is set:

configure_blueguardrails_tracer(name="support-agent", api_key="your-api-key")

Trace a pipeline

Add BlueGuardrailsConnector to your pipeline. You don't need to connect it to other components.

from haystack import Pipeline
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.utils import Secret

from blueguardrails_haystack import BlueGuardrailsConnector

pipe = Pipeline()

pipe.add_component(
    "blueguardrails",
    BlueGuardrailsConnector(
        name="support-bot",
        api_key=Secret.from_env_var("BLUE_GUARDRAILS_API_KEY"),
        tags={"environment": "development"},
    ),
)

pipe.add_component("llm", OpenAIChatGenerator(model="gpt-5.4-mini"))

result = pipe.run(
    {
        "llm": {
            "messages": [
                ChatMessage.from_user("Reply in one sentence. What is Haystack?")
            ]
        }
    }
)

print(result["llm"]["replies"][0].text)

When the pipeline runs, Blue Guardrails receives a trace for the llm call.

Load and trace a pipeline from YAML

Haystack pipelines can be serialized to and loaded from YAML. Put the connector in the YAML just like any other component; loading the pipeline initializes Blue Guardrails tracing.

pipeline.yaml:

components:
  blueguardrails:
    type: blueguardrails_haystack.components.connector.BlueGuardrailsConnector
    init_parameters:
      name: support-bot
      api_key:
        type: env_var
        env_vars:
          - BLUE_GUARDRAILS_API_KEY
        strict: true
      sample_rate: 1.0
      tags:
        environment: development
  llm:
    type: haystack.components.generators.chat.openai.OpenAIChatGenerator
    init_parameters:
      model: gpt-5.4-mini
      api_key:
        type: env_var
        env_vars:
          - OPENAI_API_KEY
        strict: true
connections: []

Set BLUE_GUARDRAILS_API_KEY and OPENAI_API_KEY in your environment, then load and run it:

from pathlib import Path

from haystack import Pipeline
from haystack.dataclasses import ChatMessage

pipeline = Pipeline.loads(Path("pipeline.yaml").read_text())

result = pipeline.run(
    {
        "llm": {
            "messages": [
                ChatMessage.from_user("Reply in one sentence. What is Haystack?")
            ]
        }
    }
)

print(result["llm"]["replies"][0].text)

Use Pipeline.dumps() or Pipeline.dump() to serialize a Python-built pipeline back to YAML.

Configure the connector

BlueGuardrailsConnector(
    name="production-rag",
    api_key=Secret.from_env_var("BLUE_GUARDRAILS_API_KEY"),
    sample_rate=0.1,
    tags={"environment": "production", "team": "search"},
)

Traces include generator inputs and outputs. Review your data handling requirements before you enable the connector in production.

License

Apache-2.0