agent-relay

Drop-in OpenAI and Anthropic API server that routes through agent CLIs (currently Claude Code).

Compatibility note: claude-relay remains available as a compatibility package/command alias.

Why

You have tools that speak the OpenAI or Anthropic API. You have Claude Code with its tools, MCP servers, and agentic capabilities. agent-relay bridges the two — point any compatible client at it and every request flows through claude -p under the hood.

• Use Claude Code from any OpenAI or Anthropic client — Cursor, Continue, aider, LangChain, custom scripts
• Keep Claude Code's superpowers — tool use, MCP servers, file access, shell execution
• Zero config — if claude works on your machine, so does this
• Real token usage — reports actual token counts from Claude (not zeros)
• Token-level streaming — uses --include-partial-messages for true real-time deltas

Install

# With uv (recommended)
uvx agent-relay serve

# Or install globally
uv tool install agentrelay-cli
agent-relay serve

# Or from source
git clone https://github.com/npow/claude-relay.git
cd claude-relay
uv sync
uv run agent-relay serve

Quick start

agent-relay serve
# Server starts on http://localhost:18082

Run as background service (macOS)

# Install and auto-start on login
agent-relay service install

The installer will offer to add these to your ~/.zshrc (or ~/.bashrc) so every SDK and agent picks up the relay automatically:

export ANTHROPIC_BASE_URL="http://127.0.0.1:18082"
export OPENAI_BASE_URL="http://127.0.0.1:18082/v1"

# Check status
agent-relay service status

# Update
uv tool upgrade agentrelay-cli
agent-relay service restart

# Stop and remove
agent-relay service uninstall

Point any OpenAI-compatible client at it:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:18082/v1", api_key="unused")

# Streaming
for chunk in client.chat.completions.create(
    model="sonnet",
    messages=[{"role": "user", "content": "Hello!"}],
    stream=True,
):
    print(chunk.choices[0].delta.content or "", end="")

# Non-streaming
resp = client.chat.completions.create(
    model="sonnet",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(resp.choices[0].message.content)

Anthropic SDK

import anthropic

# Just set the base URL — the SDK reads ANTHROPIC_BASE_URL automatically
# export ANTHROPIC_BASE_URL=http://localhost:18082
client = anthropic.Anthropic(base_url="http://localhost:18082")

# Streaming
with client.messages.stream(
    model="sonnet",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello!"}],
) as stream:
    for text in stream.text_stream:
        print(text, end="")

# Non-streaming
resp = client.messages.create(
    model="sonnet",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello!"}],
)
print(resp.content[0].text)

LangChain

from langchain_anthropic import ChatAnthropic

# export ANTHROPIC_BASE_URL=http://localhost:18082
llm = ChatAnthropic(model="sonnet")
print(llm.invoke("Hello!").content)

curl

# OpenAI format
curl http://localhost:18082/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"sonnet","messages":[{"role":"user","content":"Hello"}],"stream":true}'

# OpenAI Responses format
curl http://localhost:18082/v1/responses \
  -H "Content-Type: application/json" \
  -d '{"model":"sonnet","input":"Hello"}'

# Anthropic format
curl http://localhost:18082/v1/messages \
  -H "Content-Type: application/json" \
  -d '{"model":"sonnet","max_tokens":1024,"messages":[{"role":"user","content":"Hello"}]}'

Configuration

agent-relay serve [--host HOST] [--port PORT]

API

All endpoints also work without the /v1 prefix. CORS is enabled for all origins.

Observability

Request logging is enabled by default with:

• request method, path, status, and duration
• per-request X-Request-Id header (accepts incoming x-request-id too)
• in-process counters exposed at /metrics (and /v1/metrics)

Environment variables:

Install Sentry support:

uv pip install 'agentrelay-cli[observability]'

Crash detection tip: monitor /health fields pid and started_at; if either changes unexpectedly, the process restarted.

Minimal setup example:

export SENTRY_DSN="https://<key>@o<org>.ingest.sentry.io/<project>"
export SENTRY_ENVIRONMENT="prod"
export SENTRY_TRACES_SAMPLE_RATE="0.1"
export SENTRY_LOG_LEVEL="INFO"
export SENTRY_EVENT_LEVEL="ERROR"
agent-relay serve

Better Stack logs

Install observability extras:

uv pip install 'agentrelay-cli[observability]'

Set env vars and run:

export BETTERSTACK_SOURCE_TOKEN="<your-source-token>"
export BETTERSTACK_INGESTING_HOST="https://in.logs.betterstack.com"
export BETTERSTACK_LOG_LEVEL="INFO"
agent-relay serve

Then open Better Stack Logs and filter by service=claude-relay or request_id.

Smart model routing

Use model: "auto" (or enable global routing) to let relay choose a model by prompt complexity:

export AGENT_RELAY_SMART_ROUTING=true
export AGENT_RELAY_ROUTING_DEFAULT_MODEL=sonnet
agent-relay serve

Behavior:

• routing is LLM-first (local classifier), no rule fallback path
• explicit model choice is preserved unless AGENT_RELAY_SMART_ROUTING_FORCE=true
• if classifier fails/times out, request returns routing_error (HTTP 503)

Strategy notes:

• tier classifier returns exactly one tier: SIMPLE/MEDIUM/COMPLEX/REASONING
• tier maps to configurable models via env vars

Ship local logs to remote Grafana/Loki (with reconnect)

If agent-relay runs on your laptop and Grafana/Loki is on another host (for example hetzner-recon), use the included scripts:

Terminal 1:

./scripts/run-relay-with-logfile.sh

Terminal 2:

./scripts/run-promtail-shipper.sh hetzner-recon

What this does:

• keeps a local durable log file at ~/.claude-relay-observability/logs/relay.log
• starts an SSH tunnel to remote Loki (127.0.0.1:3100 on the remote host)
• runs Promtail in Docker to ship logs to Loki through the tunnel
• auto-reconnects the tunnel if network drops, then Promtail resumes sending

Grafana Explore query:

{job="claude-relay", service="claude-relay"}

Supported features

Models

Pass any model name — it goes directly to claude --model:

Limitations

• temperature, max_tokens, top_p, and other sampling parameters are ignored (Claude Code CLI does not expose them)
• No tool/function calling passthrough (Claude Code uses its own tools internally, but they aren't exposed via the OpenAI tool-calling protocol)
• Each request spawns a new claude process (~2-3s overhead on top of API latency)
• No image/audio content forwarding — only text parts of multimodal messages are extracted

How it works

OpenAI client     ─┐
                    ├→  claude-relay  →  claude -p  →  Anthropic API
Anthropic client  ─┘     (FastAPI)      (stream-json)

Each request spawns a claude -p process with --output-format stream-json --include-partial-messages. The proxy translates between the OpenAI or Anthropic wire format and Claude Code's streaming JSON protocol. Requests are stateless — no conversation history bleeds between calls.

Development

uv sync
uv run pytest tests/ -v

License

MIT