relayai-gateway

OpenAI and Anthropic compatible API, with tool calling, for the models in opencode.

Install it, and every tool that speaks the OpenAI or Anthropic API (the openai SDK, LangChain, Open WebUI, Claude Code, your own scripts) can use the models you have in opencode, including opencode's free models. relayai finds opencode, runs opencode serve in the background for you, and translates between the APIs.

• POST /v1/chat/completions (OpenAI) and POST /v1/messages (Anthropic), streaming included
• Tool calling on both, so agent harnesses like Claude Code work
• Models are read live from opencode, nothing is hardcoded
• A CLI (relayai), a server, and a Python SDK in one package with no dependencies
• The same tool exists for Node: npm install -g relayai-gateway

pip install relayai-gateway
relayai chat "Explain DNS in one sentence"

Contents

• Requirements
• Quick start
• Use it from any OpenAI or Anthropic client
• Full API reference
• Claude Code
• Python SDK
• CLI reference
• Configuration
• How it works
• Which models work with tools
• Limits
• Security
• Troubleshooting

Requirements

• Python 3.8 or newer

• opencode (1.x or 2.x). If you do not have it:

  relayai install-opencode        # runs: npm install -g @opencode/cli (asks first)
  

  or install it yourself as described at https://opencode.ai/docs.

• At least one model in opencode. opencode's free models work out of the box; to add your own providers (OpenRouter, Anthropic, OpenAI, Groq, ...) run opencode auth login.

Check everything with:

relayai doctor --test

Quick start

relayai start                 # opencode serve + gateway in the background
relayai models                # what you can use
relayai chat "Hello"          # one question
relayai chat                  # interactive chat (/reset clears it)
relayai status                # URLs and API key
relayai stop                  # stops the gateway and the opencode serve it started

relayai serve does the same as start but in the foreground, with request logs; Ctrl+C stops everything.

The gateway listens on http://127.0.0.1:8000. Its API key is created once and stored in ~/.relayai/api_key; relayai status shows it.

Commands like chat and models use the background gateway when it runs. Without it they start a private one for the duration of the command, which takes a few seconds longer.

Use it from any OpenAI or Anthropic client

relayai env prints the variables for your shell:

eval "$(relayai env)"                     # bash / zsh
relayai env | Invoke-Expression           # PowerShell

curl:

curl http://127.0.0.1:8000/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" -H "Content-Type: application/json" \
  -d '{"model": "default", "messages": [{"role": "user", "content": "Say hi"}]}'

The official openai package:

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="rai-...")
reply = client.chat.completions.create(
    model="opencode/big-pickle",
    messages=[{"role": "user", "content": "Hello"}],
)
print(reply.choices[0].message.content)

Tools work the normal OpenAI way (tools=[...], then tool_calls in the reply, then a role: "tool" message with the result).

Full API reference

Every endpoint, request field, streaming event, error code and client integration (OpenAI and Anthropic SDKs for Python and Node, LangChain, Vercel AI SDK, Open WebUI, Claude Code), plus HTTPS, network access and running on boot, is documented in the API reference.

Claude Code

relayai claude                                   # interactive, through the gateway
relayai claude -m opencode/big-pickle            # pick the model
relayai claude -- -p "list the files here"       # anything after -- goes to claude

This sets ANTHROPIC_BASE_URL, ANTHROPIC_AUTH_TOKEN and the model variables for that one Claude Code process only; your normal claude is untouched. Pick a model from the tools table, because Claude Code depends on tool calling.

Python SDK

from relayai import RelayAI

with RelayAI() as ai:                       # finds/starts opencode and a private gateway
    print(ai.chat("Hello"))                 # -> str
    for piece in ai.stream("Tell me a joke"):
        print(piece, end="", flush=True)
    print(ai.models())                      # -> ["opencode/big-pickle", ...]

RelayAI reuses the background gateway from relayai start when it runs (pass reuse=False to always run a private one). On exit it stops whatever it started.

Options are keywords: RelayAI(model="opencode/big-pickle", timeout=120, opencode_url="http://127.0.0.1:4096", opencode_password="..."). Every field of Config works.

Conversations, system prompt, extra parameters

ai.chat([
    {"role": "user", "content": "My name is Asha."},
    {"role": "assistant", "content": "Nice to meet you, Asha."},
    {"role": "user", "content": "What is my name?"},
], system="Answer in one word.", model="opencode/big-pickle")

Tool calling

weather = {"type": "function", "function": {
    "name": "get_weather",
    "description": "Current weather for a city",
    "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]},
}}

messages = [{"role": "user", "content": "Weather in Paris?"}]
reply = ai.complete(messages=messages, tools=[weather])          # raw OpenAI response dict
call = reply["choices"][0]["message"]["tool_calls"][0]
messages += [reply["choices"][0]["message"],
             {"role": "tool", "tool_call_id": call["id"], "content": "18C, sunny"}]
print(ai.chat(messages, tools=[weather]))

Anthropic format: ai.messages(messages=[...], tools=[...]) returns the Anthropic response dict.

Plug the official SDKs in: ai.base_url (OpenAI, ends in /v1), ai.anthropic_base_url and ai.api_key.

Errors: every failure raises relayai.ApiError with .status (HTTP status), .message, .kind and .code.

Lower level pieces

from relayai import Config, Gateway

with Gateway(Config.from_env(api_key="secret", port=9000)) as gw:
    print(gw.base_url)        # http://127.0.0.1:9000/v1, serving in a background thread
    input("Enter to stop")

CLI reference

Every command accepts the options in the next section. relayai <command> --help shows them.

Configuration

Command line flags win over environment variables, which win over defaults.

In code: Config.from_env(port=9000, model="opencode/big-pickle").

How it works

opencode management. Unless you pass --opencode-url, relayai looks for the opencode executable, starts opencode serve on 127.0.0.1 with a freshly generated password, and keeps it in ~/.relayai/workspace so it never works inside your projects. Its details go in ~/.relayai/opencode.json, so later commands (and the Node package) reuse it. relayai stop, serve on exit and the SDK on close stop it again if they started it.

Requests. Every API request becomes one short-lived opencode session that is deleted afterwards. The OpenAI and Anthropic APIs are stateless, so the whole conversation is folded into one prompt. opencode's own tools (shell, file editing) never run.

Tool calling is emulated: your tools are described in the prompt, the model answers with <tool_call>{"name": ..., "arguments": ...}</tool_call> blocks, and relayai turns those into OpenAI tool_calls or Anthropic tool_use. Your program runs the tools, exactly as with a native API.

Files in ~/.relayai: api_key, gateway.json and opencode.json (what is running), gateway.log and opencode.log, and workspace/.

Which models work with tools

Plain chat works with every model opencode can reach. Emulated tool calling depends on the model following the <tool_call> format. Tested on 2026-11-11 with opencode 2.0.31's free models:

The models marked "no" try to use opencode's own built-in tools instead of writing text, and the reply comes back empty ("The model returned no text"). Strong paid models generally follow the format well. Free model line-ups change often, so test yours with a small tools request before relying on it.

Limits

• Streaming arrives in one piece. opencode returns the finished answer, so stream: true sends it in chunks once it is ready (with keepalives meanwhile), not token by token.
• Text only. Images and documents are replaced by a placeholder.
• Token counts are estimates on opencode 2.x (about 4 characters per token); opencode 1.x reports real counts.
• Endpoints: chat completions, messages, models, count_tokens (an estimate). No embeddings, no Responses API, no images.
• Free models are rate limited by opencode; bursts of parallel requests get "Free usage limit reached" for a while.

Security

• The gateway listens on 127.0.0.1 only. --host 0.0.0.0 exposes it to your network; anyone with the key can then spend your providers' quota, so use a long key.
• Every API request needs the key (Authorization: Bearer ... or x-api-key). /health is the only open endpoint.
• The key and the opencode password are stored in ~/.relayai with owner-only permissions on Linux and macOS.

Troubleshooting

Logs: ~/.relayai/gateway.log (background gateway) and ~/.relayai/opencode.log.

Changelog and contributing

Versions are date based (YYYY.M.D, the release day). See the changelog, contributing guide and code of conduct.

License

MIT, by Mithun Gowda B. Source: github.com/mithun50/relayai-gateway