GatewaySDK

AI Agent Training and Evaluation Platform

GatewaySDK is a comprehensive toolkit for training and evaluating AI agents using reinforcement learning, automatic prompt optimization, and supervised fine-tuning.

Core Features

• Turn your agent into an optimizable beast with minimal code changes
• Build with any agent framework (LangChain, OpenAI Agent SDK, AutoGen, CrewAI, and more)
• Selectively optimize one or more agents in a multi-agent system
• Embraces algorithms like Reinforcement Learning, Automatic Prompt Optimization, Supervised Fine-tuning and more

Installation

pip install gatewaysdk

For optional dependencies:

# For LLMProxy feature (note: has strict dependency pins for boto3, grpcio, uvicorn)
pip install 'gatewaysdk[proxy]'

# For Trainer (includes the proxy runtime it composes)
pip install 'gatewaysdk[trainer]'

# For GPU statistics in runner heartbeat snapshots
pip install 'gatewaysdk[gpu-monitoring]'

# For APO (Automatic Prompt Optimization)
pip install 'gatewaysdk[apo]'

# For VERL integration
pip install 'gatewaysdk[verl]'

# For Weave integration
pip install 'gatewaysdk[weave]'

# For MongoDB store
pip install 'gatewaysdk[mongo]'

Quick Start

from gatewaysdk import GatewayIngestionClient

# Reads GATEWAY_HOST, GATEWAY_PUBLIC_KEY, and GATEWAY_SECRET_KEY.
client = GatewayIngestionClient.from_env()

# Your agent code here...

Gateway Ingestion Helpers

Use these helpers when you want to send external agent traces/scores directly to a Gateway instance.

1) Gateway ingestion client

from gatewaysdk import GatewayIngestionClient, GatewayIngestionConfig

client = GatewayIngestionClient(
    GatewayIngestionConfig(
        host="http://localhost:3000",
        public_key="pk-lf-...",
        secret_key="sk-lf-...",
    )
)

# Send trace event
client.ingest_trace_event(
    trace_id="trace-001",
    name="Agent Trace",
    input_data={"prompt": "hello"},
    output_data={"answer": "world"},
    session_id="session-001",
    environment="default",
)

# Send scores efficiently in batches
client.send_scores(
    [
        {"name": "task_success", "value": 0.95, "dataType": "NUMERIC", "traceId": "trace-001"},
        {"name": "safety", "value": 1, "dataType": "BOOLEAN", "traceId": "trace-001"},
    ],
    mode="ingestion-batch",
)

2) Simple API wrapper sample (dependency-free)

For a minimal stdlib-only helper and examples, see: samples/gateway-agent-trace-wrapper/.

Security Wrapper

Use the security wrapper when Gateway should decide which native tools an agent can see and execute. Your framework keeps owning the tools; Gateway resolves policy, hides denied tools in filter/enforce modes, authorizes before execution, and stamps OpenTelemetry security attributes when tracing is active.

from gatewaysdk import GatewaySecurity

gw = GatewaySecurity.from_env(
    agent={
        "id": "support-triage",
        "name": "Support Triage",
        "version": "1.4.2",
    }
)

def search_tickets(query: str) -> list[dict]:
    return zendesk.search(query)

def delete_ticket(ticket_id: str) -> dict:
    return zendesk.delete(ticket_id)

tools = gw.secure_tools(
    toolset_id="zendesk",
    provider="zendesk",
    tools=[search_tickets, delete_ticket],
    context={
        "user_id": current_user.id,
        "team_id": "support",
        "environment": "production",
        "trace_id": trace_id,
        "session_id": session_id,
    },
)

CLI Usage

# Start the GatewaySDK server
msk store serve

# Run with vLLM
msk vllm start

Documentation

For full documentation, visit https://docs.gateway.dev/gatewaysdk/

License

MIT License - see LICENSE for details.