agent-status-sdk 1.0.0

Agent Status SDK - Outside-in monitoring for AI agents

Agent Status SDK

Outside-in monitoring for AI agents. Monitor any HTTP-accessible AI agent from distributed residential endpoints worldwide.

Installation

pip install agent-status-sdk

# With LangChain integration
pip install agent-status-sdk[langchain]

# All integrations
pip install agent-status-sdk[all]

Quick Start

import agent_status

# Initialize with your API key
agent_status.init(api_key="rora_xxx")

# Register an agent for continuous monitoring
agent = agent_status.register(
    endpoint="https://api.mycompany.com/chat",
    name="Support Bot",
    interval_minutes=60,  # Check every hour
)

print(f"Registered: {agent.id}")

# Check current status
status = agent_status.status(agent.id)
print(f"Verdict: {status.verdict}")  # UP, DEGRADED, DOWN
print(f"Uptime: {status.uptime_24h}%")
print(f"Latency: {status.latency_p95}ms")

One-Off Validation

Run a quick validation without registering for continuous monitoring:

result = agent_status.run(
    endpoint="https://api.example.com/chat",
    prompts=["What is 2+2?", "Hello!"],
)

print(f"Verdict: {result.verdict}")
print(f"P95 Latency: {result.latency_p95}ms")
print(f"Pass Rate: {result.pass_rate}")

Authentication

For agents requiring authentication:

# Bearer token
agent = agent_status.register(
    endpoint="https://api.mycompany.com/chat",
    name="Private Bot",
    auth={"type": "bearer", "token": "sk-xxx"},
)

# API key in header
agent = agent_status.register(
    endpoint="https://api.mycompany.com/chat",
    name="API Bot",
    auth={"type": "api_key", "header": "X-API-Key", "value": "xxx"},
)

Advanced Options

agent = agent_status.register(
    endpoint="https://api.mycompany.com/chat",
    name="Enterprise Bot",

    # Probing configuration
    interval_minutes=60,       # How often to probe
    max_nodes_per_run=10,      # Distributed nodes per check
    geos=["us", "eu", "ap"],   # Geographic regions
    timeout_ms=30000,          # Request timeout

    # Validation options
    eval_type="llm_judge",     # "basic", "llm_judge", or "all"
    gold_prompt_profile="search_agent",  # Specialized prompts
    inject_geo_context=True,   # Add location to prompts

    # Response handling
    streaming=True,            # SSE streaming responses
)

LangChain Integration

Expose for Outside-In Monitoring (Recommended)

The easiest way to monitor a LangChain agent - no infrastructure needed:

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from agent_status.integrations.langchain import expose

# Build your chain
prompt = ChatPromptTemplate.from_template("You are a helpful assistant. {input}")
llm = ChatOpenAI()
chain = prompt | llm

# Expose to the distributed monitoring network - one line!
expose(chain, api_key="fb_live_xxx", agent_name="My Support Bot")

# That's it! Nodes around the world will now probe your agent.
# Output:
# ============================================================
#   Agent Status - Agent Exposed!
#   Name: My Support Bot
#   URL:  https://rora-tunnel.carmel.so/probe/agent-xxx
#   Press Ctrl+C to stop
# ============================================================

This works even for agents running locally on your laptop - no need to deploy anywhere!

Non-Blocking Mode

Run in background while your app does other things:

url = expose(
    chain,
    api_key="fb_live_xxx",
    agent_name="Background Bot",
    blocking=False,  # Returns immediately
)
print(f"Agent exposed at: {url}")

# Your app continues running...

Local Callback Handler

For tracking metrics locally without outside-in monitoring:

from langchain_openai import ChatOpenAI
from agent_status.integrations.langchain import AgentStatusCallbackHandler

handler = AgentStatusCallbackHandler(
    api_key="fb_live_xxx",
    agent_name="My LangChain Agent",
)

llm = ChatOpenAI(callbacks=[handler])
result = llm.invoke("Hello!")

print(handler.metrics)
# {'total_calls': 1, 'llm_calls': 1, 'total_latency_ms': 523, ...}

Tracked Metrics (Callback Handler)

The callback handler tracks:

• LLM calls (latency, tokens, model)
• Chain executions
• Tool/function calls
• Errors and exceptions

CLI Usage

# Set your API key
export RORA_API_KEY=rora_xxx

# Check agent status
agent-status status <agent_id>

# Run one-off validation
agent-status run https://api.example.com/chat --prompts "Hello,How are you?"

# List all agents
agent-status list

# Register a new agent
agent-status register https://api.mycompany.com/chat --name "My Bot"

# Delete an agent
agent-status delete <agent_id>

Gold Prompt Profiles

Agent Status uses "gold prompts" - carefully crafted test prompts for different agent types:

Profile	Description
general	Generic conversational prompts
search_agent	Web search and information retrieval
code_generator	Code generation and debugging
data_retriever	Database and API queries
customer_support	Support and FAQ handling
creative_writer	Content generation

Evaluation Types

Type	Description
basic	Response format and latency checks
llm_judge	GPT-4 evaluates response quality
all	Both basic and LLM evaluation

Response Models

Agent

agent.id              # UUID
agent.name            # Display name
agent.endpoint_url    # HTTP endpoint
agent.status          # active, paused, deleted
agent.last_status     # UP, DEGRADED, DOWN

AgentStatus

status.verdict       # UP, DEGRADED, DOWN, UNKNOWN
status.uptime_24h    # 24-hour uptime percentage
status.uptime_7d     # 7-day uptime percentage
status.latency_p50   # P50 latency (ms)
status.latency_p95   # P95 latency (ms)
status.pass_rate     # Pass rate (0-1)
status.total_checks  # Total probes run

RunResult

result.verdict          # UP, DEGRADED, DOWN
result.latency_p50      # P50 latency (ms)
result.latency_p95      # P95 latency (ms)
result.pass_rate        # Pass rate (0-1)
result.total_probes     # Probes sent
result.successful_probes  # Successful probes
result.by_region        # Per-region breakdown
result.judge_result     # LLM evaluation (if enabled)

Error Handling

from agent_status.client import AgentStatusError, AgentStatusAuthError, AgentStatusNotFoundError

try:
    status = agent_status.status("invalid-id")
except AgentStatusNotFoundError:
    print("Agent not found")
except AgentStatusAuthError:
    print("Invalid API key")
except AgentStatusError as e:
    print(f"Error: {e}")

Environment Variables

Variable	Description
RORA_API_KEY	Your API key (required for CLI)
RORA_BASE_URL	API base URL (optional, for testing)

Note: Environment variables retain the RORA_ prefix for backward compatibility.

Links

• Homepage
• Dashboard
• GitHub

License

MIT