ai-relay

WebSocket relay that bridges AI coding agent CLIs (Claude Code, Codex, Gemini CLI, Snowflake Cortex, and more) to any web interface — stream reasoning, tool calls, and file changes in real time.

Above: Claude Code, driven through ai-relay, builds and renders a live interactive dashboard in a browser — reasoning, tool calls, and the finished artifact streamed in real time. ▶ Watch the full walkthrough

Install

pip install ai-relay

Quick start

One-shot mode (local dev)

# Start the relay server (default: ws://0.0.0.0:8765)
ai-relay --port 8765

Server mode (container / daemon)

# Persistent server — each WebSocket connection becomes one independent agent session
ai-relay serve --port 9000

Then connect from OhWise Lab (or any WebSocket client) and send a handshake:

{"tool": "claude", "folder": "/path/to/project", "model": "claude-sonnet-4-6"}

The relay streams structured JSON events over WebSocket and forwards your messages to the selected backend. Claude Code and Gemini use native JSONL process protocols, Codex uses the app-server JSON-RPC protocol, and Snowflake Cortex uses HTTP/SSE. The PTY bridge is retained only for generic/legacy CLI tools.

Running in a container

ai-relay serve is designed to run inside a Docker container as a persistent daemon:

FROM python:3.11-slim
RUN pip install ai-relay
# Install your AI CLI here (e.g. npm install -g @anthropic-ai/claude-code)
CMD ["ai-relay", "serve", "--port", "9000"]

Each incoming WebSocket connection spawns an independent agent session. Multiple clients can connect simultaneously.

Recommended architecture (enterprise-grade SaaS)

For a multi-tenant, production SaaS, the recommended design is:

• One isolated compute boundary per tenant/user (the isolation boundary) — use whatever fits your needs: a VM, pod, container, serverless/Lambda sandbox, dedicated server, or on-prem host. Each runs one ai-relay serve that multiplexes all of that tenant's sessions — one WebSocket connection per session_id. Never share one runtime across tenants, and do not start one process per session.
• A server-side orchestrator/hub in front (never expose ai-relay to the browser): it holds one WebSocket per session to the relay, fans out to N browser tabs, enforces auth/RBAC, persists conversation history + state in a database, and keeps the connection open so in-progress turns survive a browser refresh.
• Per-tenant filesystem isolation + dropped capabilities + internal-only networking.

ai-relay is the stateless streaming engine; durability and multi-tenancy live in your orchestrator. Full reference architecture and rationale: docs/architecture.md → Session multiplexing model / Recommended architecture.

Event types

Sending commands

Send JSON over WebSocket:

{"text": "refactor the authentication module to use JWT"}

Claude Code also accepts structured web-client messages:

{"type": "user_message", "content": "refactor the authentication module to use JWT"}

Permission responses:

{"type": "permission_response", "request_id": "req", "behavior": "allow", "updatedInput": {"command": "git status"}}

Codex permission responses can also use:

{"type": "permission_response", "request_id": "req", "allow": true}

Interrupt the active structured turn:

{"type": "interrupt"}

Relay-handled slash commands (Claude per-turn runtime)

In --print / stream-json mode the CLI has no interactive TUI, so the relay intercepts a few commands and answers them itself (they are never forwarded to the subprocess):

Auto-reset safeguards (resume durability)

Claude Code resumes a conversation by reloading its on-disk transcript each turn. A very large transcript makes --resume slow or hang. The relay guards against this automatically:

• Transcript-size rotation — before resuming, if the transcript exceeds AI_RELAY_TRANSCRIPT_ROTATE_BYTES (default 5 MB) the relay starts a fresh conversation and emits a response (metadata.source = "auto_rotate").
• Resume-hang watchdog — if a resumed turn emits system/init then stalls for AI_RELAY_RESUME_HANG_TIMEOUT seconds (default 90) with no result, the relay kills the subprocess and retries once without --resume (metadata.source = "resume_hang_retry").

Earlier conversation history shown in your own UI is unaffected — rotation only resets the CLI's resume context, not your application's stored history.

Codex uses codex app-server --listen stdio:// and keeps a persistent thread behind the WebSocket session:

{"tool": "codex", "folder": "/path/to/project", "model": "gpt-5.2"}

Gemini CLI uses ACP (Agent Communication Protocol) mode. Each text message starts one Gemini turn:

{"tool": "gemini", "folder": "/path/to/project", "model": "gemini-2.5-flash"}

Gemini-specific behavior — per-turn process lifecycle

Unlike Claude Code (persistent process + --resume) and Codex (persistent app-server), Gemini CLI ACP mode (≥ 0.40.x) exits the subprocess after every session/prompt turn completes (exit code 0). This is by design in the Gemini CLI implementation.

The relay handles this transparently:

• On the first turn the relay calls session/new and saves the returned sessionId to .gemini_acp_session in the session working folder.
• On each subsequent reconnect the relay calls session/load with that ID so Gemini restores conversation context in the new process.
• If session/load fails (session expired or folder was wiped) the relay falls back to session/new automatically.

Implication for permission approvals: because the process exits after each turn, permission responses must be sent while the current turn's process is still alive (i.e. before the session/prompt RPC completes). The default session/prompt timeout is 5 minutes, which covers typical interactive permission dialogs.

Snowflake Cortex uses API configuration in the handshake.

Cortex chat mode:

{
  "tool": "cortex",
  "mode": "chat",
  "model": "claude-sonnet-4-5",
  "snowflake": {
    "account_url": "https://<account>.snowflakecomputing.com",
    "token_env": "SNOWFLAKE_PAT"
  }
}

Cortex Analyst mode:

{
  "tool": "cortex",
  "mode": "analyst",
  "snowflake": {
    "account_url": "https://<account>.snowflakecomputing.com",
    "token_env": "SNOWFLAKE_PAT",
    "semantic_view": "DB.SCHEMA.VIEW"
  }
}

To send CLI commands (e.g. /compact, /clear):

{"text": "/compact"}

Supported tools

Configuration reference (ohwise-lab-ctrl)

When using ohwise-lab alongside ai-relay, all settings are controlled via environment variables — nothing is hardcoded:

Python API

from ai_relay import RelayServer

server = RelayServer(host="0.0.0.0", port=8765)
server.run()

Documentation

• Architecture & Developer Guide — internals, event system, auth, turn lifecycle, reconnect handling, adding providers, containerised deployment

Changelog

See CHANGELOG.md for release notes.

License

Apache-2.0