GatewaySDK

AI Agent Training and Evaluation Platform

GatewaySDK is a comprehensive toolkit for training and evaluating AI agents using reinforcement learning, automatic prompt optimization, and supervised fine-tuning.

Core Features

• Turn your agent into an optimizable beast with minimal code changes
• Build with any agent framework (LangChain, OpenAI Agent SDK, AutoGen, CrewAI, and more)
• Selectively optimize one or more agents in a multi-agent system
• Embraces algorithms like Reinforcement Learning, Automatic Prompt Optimization, Supervised Fine-tuning and more

Installation

gatewaysdk 0.4.0 and later require Python 3.12 or newer (0.3.x ran on 3.10).

pip install gatewaysdk

For optional dependencies:

# For LLMProxy feature (note: has strict dependency pins for boto3, grpcio, uvicorn)
pip install 'gatewaysdk[proxy]'

# For Trainer (includes the proxy runtime it composes)
pip install 'gatewaysdk[trainer]'

# For GPU statistics in runner heartbeat snapshots
pip install 'gatewaysdk[gpu-monitoring]'

# For APO (Automatic Prompt Optimization)
pip install 'gatewaysdk[apo]'

# For VERL integration
pip install 'gatewaysdk[verl]'

# For Weave integration
pip install 'gatewaysdk[weave]'

# For MongoDB store
pip install 'gatewaysdk[mongo]'

# For auto-instrumentation (OpenAI, OpenAI Agents, Anthropic, LiteLLM via OpenInference)
pip install 'gatewaysdk[tracing]'

Quick Start

from gatewaysdk import GatewayIngestionClient

# Reads GATEWAY_HOST, GATEWAY_PUBLIC_KEY, and GATEWAY_SECRET_KEY.
client = GatewayIngestionClient.from_env()

# Your agent code here...

Gateway Ingestion Helpers

Use these helpers when you want to send external agent traces/scores directly to a Gateway instance.

1) Gateway ingestion client

from gatewaysdk import GatewayIngestionClient, GatewayIngestionConfig

client = GatewayIngestionClient(
    GatewayIngestionConfig(
        host="http://localhost:3000",
        public_key="pk-lf-...",
        secret_key="sk-lf-...",
    )
)

# Send trace event
client.ingest_trace_event(
    trace_id="trace-001",
    name="Agent Trace",
    input_data={"prompt": "hello"},
    output_data={"answer": "world"},
    session_id="session-001",
    environment="default",
)

# Send scores efficiently in batches
client.send_scores(
    [
        {"name": "task_success", "value": 0.95, "dataType": "NUMERIC", "traceId": "trace-001"},
        {"name": "safety", "value": 1, "dataType": "BOOLEAN", "traceId": "trace-001"},
    ],
    mode="ingestion-batch",
)

2) Simple API wrapper sample (dependency-free)

For a minimal stdlib-only helper and examples, see: samples/gateway-agent-trace-wrapper/.

Security Wrapper

Use the security wrapper when Gateway should decide which native tools an agent can see and execute. Your framework keeps owning the tools; Gateway resolves policy, hides denied tools in filter/enforce modes, authorizes before execution, and stamps OpenTelemetry security attributes when tracing is active.

from gatewaysdk import GatewaySecurity

gw = GatewaySecurity.from_env(
    agent={
        "id": "support-triage",
        "name": "Support Triage",
        "version": "1.4.2",
    }
)

def search_tickets(query: str) -> list[dict]:
    return zendesk.search(query)

def delete_ticket(ticket_id: str) -> dict:
    return zendesk.delete(ticket_id)

tools = gw.secure_tools(
    toolset_id="zendesk",
    provider="zendesk",
    tools=[search_tickets, delete_ticket],
    context={
        "user_id": current_user.id,
        "team_id": "support",
        "environment": "production",
        "trace_id": trace_id,
        "session_id": session_id,
    },
)

The gateway CLI

pip install gatewaysdk installs gateway, the Python twin of the npm CLI (npm i -g @withgateway/sdk): the same command paths and JSON output, with the few one-sided commands refusing by name.

export GATEWAY_HOST=https://withgateway.ai
export GATEWAY_PUBLIC_KEY=pk-lf-example
gateway auth login     # reads GATEWAY_HOST / GATEWAY_PUBLIC_KEY / GATEWAY_SECRET_KEY, prompts for the rest
gateway auth status    # the credentials in effect: the environment first, then the stored file
gateway --help         # every group

auth login writes ~/.gateway/config.json; GATEWAY_CONFIG_DIR moves it, and GATEWAY_API_KEY is accepted in place of GATEWAY_SECRET_KEY. The SDK's platform clients (FilesClient.from_env(), WorldDataClient.from_env(), world sessions, sandboxes, connectors, environments) use the same login for any value the environment leaves unset, but send its keys only to the host it was stored for, and raise CredentialsNotConfigured when there is none. The agent-runtime clients (personas, users, memory, context, security, tool access) read the environment only.

The msk CLI

# Start the GatewaySDK server
msk store serve

# Run with vLLM
msk vllm start

Documentation

For full documentation, visit https://withgateway.ai/docs

License

MIT License - see LICENSE for details.