aw-sdk 0.1.1

Lightweight drop-in wrapper for the OpenAI Python client that logs telemetry and detects PII risks.

AgentWatch: Proactive LLM Governance Platform

AgentWatch is an ultra-low latency API proxy and SDK designed to solve the "runaway agent" problem for enterprise engineering teams. It intercepts, manages, and enforces budget constraints on LLM API requests at the edge, acting as a proactive governance layer between your application and upstream providers like OpenAI and Anthropic.

The Problem It Solves

As engineering teams adopt autonomous LLM agents (e.g., coding assistants, research bots, recursive planners), they face a critical financial vulnerability: the runaway loop.

If an agent gets stuck in a recursive error-correction loop, it can execute hundreds of API calls per minute. Because each iteration typically appends the previous output to the context window, the token size grows quadratically. This can result in a single stuck agent burning thousands of dollars in minutes—a scenario that passive monitoring tools will only report after the budget is already gone.

AgentWatch was rebuilt from the ground up to prevent this.

Core Features

1. Session-Aware Identity Tracking

Instead of treating API requests as isolated events, AgentWatch tracks iterative agent loops as Sessions.

• Every request is tagged with a session_id and an iteration_index.
• The cumulative token count for a session is securely computed and maintained server-side on Cloudflare KV. This ensures that even if a local agent process crashes, restarts, or runs in parallel, the session's financial state cannot be bypassed or reset.

2. Synchronous Pre-Call Budget Enforcement

AgentWatch acts as a strict financial gatekeeper for agent sessions.

• Developers define a budget ceiling (e.g., $2.00) per session via the AgentWatch Python SDK.
• Before any upstream LLM call is made, the SDK performs a sub-millisecond synchronous pre-flight check to the Edge Proxy (GET /v1/budget-check).
• If the session's cumulative token cost exceeds the limit, the SDK instantly blocks the execution and raises an AgentBudgetExceeded exception.
• Fail-Open Resilience: By default, if the AgentWatch proxy experiences downtime, the budget check silently fails open. This ensures our infrastructure never causes a hard outage for your production traffic.

3. Inline Anomaly Detection

AgentWatch heuristically detects runaway behavior before the budget is even exhausted.

• The Cloudflare Edge Worker maintains a rolling window of the last 5 iterations for every active session inside Cloudflare KV.
• It calculates the token growth ratio synchronously on the POST /v1/ingest handler.
• If three consecutive iterations show a >1.4x prompt growth—a hallmark signature of a context-appending loop—it asynchronously fires a Slack webhook alert via ctx.waitUntil(), adding zero latency to the critical API path.

4. Zero-Latency Proxying & Resilient Telemetry

• Ultra-Low Latency: The hot path of the proxy only handles authentication, routing, and credential rewriting.
• Asynchronous Telemetry: Payload logging and risk scanning are offloaded to background execution. The client receives the provider's response immediately.
• Cloudflare Queues: Telemetry data is pushed to a highly-available Cloudflare Queue before being batch-inserted into Supabase Postgres. This guarantees telemetry delivery even if the database goes down.

Routes

The proxy mirrors provider API paths under /v1/proxy/:provider/*.

POST /v1/proxy/openai/chat/completions
  -> https://api.openai.com/v1/chat/completions

POST /v1/proxy/anthropic/messages
  -> https://api.anthropic.com/v1/messages

Authentication

Clients authenticate to AgentWatch with a bearer token:

Authorization: Bearer aw_test_token

The Worker maps that token to a tenant ID with TENANT_TOKEN_MAP.

{
  "aw_test_token": "tenant_test"
}

The client token is never forwarded upstream. AgentWatch replaces it with the configured OpenAI or Anthropic provider key.

Required Secrets

Configure secrets before deploying:

wrangler secret put OPENAI_API_KEY
wrangler secret put ANTHROPIC_API_KEY
wrangler secret put SUPABASE_SERVICE_ROLE_KEY
wrangler secret put TENANT_TOKEN_MAP
wrangler secret put SLACK_WEBHOOK_URL

Configure non-secret values in wrangler.toml:

SUPABASE_URL = "https://YOUR_PROJECT.supabase.co"
ANTHROPIC_VERSION = "2023-06-01"

Supabase Setup

Run supabase/schema.sql and supabase/session_tracking.sql in the Supabase SQL editor. Enable the retention policy by running supabase/retention_cron.sql.

Python SDK Integration

AgentWatch integrates seamlessly via composition with standard OpenAI client wrappers:

from agentwatch import WatchedOpenAI

client = WatchedOpenAI(
    agentwatch_api_key="your_aw_key",
    agentwatch_project="checkout-service",
    agentwatch_team="payments-eng",
    agentwatch_session_id="ci-run-123",
    agentwatch_session_budget_usd=2.00,  # Strict $2 limit
    agentwatch_enforcement_mode=True
)

# Standard OpenAI API usage
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Refactor this module..."}]
)

Local Development

Install dependencies:

npm install

Run the Worker locally:

npm run dev

Typecheck:

npm run typecheck

Deploy:

npm run deploy