Strands-SGLang

Agentic RL, done correctly.

• Strands Agents SDK: a harness builder whose event-based hooks make the agent loop fully customizable.
• SGLang: a high-performance serving framework for fast, high-concurrency rollouts that exposes per-token metadata.

Strands-SGLang bridges the two so multi-turn rollouts stay on-policy — token-in, token-out, no silent retokenization drift.

Features

• Token-In/Token-Out rollouts: model-generated tokens carried through as-is
  • Only new messages are tokenized each turn with incremental chat templating
• Strict tool-call parsing: parsed exactly as generated, no heuristic repair
• Harness customization: pass tools and hooks into Agent to customize your harness
• Native SGLang /generate endpoint: high-throughput, non-streaming rollouts

For RL environment integration, please refer to strands-env

Installation

pip install strands-sglang

Or install from source with development dependencies:

git clone https://github.com/strands-rl/strands-sglang.git
cd strands-sglang
uv sync

Quick Start

1. Start SGLang Server

python -m sglang.launch_server --model-path Qwen/Qwen3.5-4B

2. Run an Agent

import asyncio
from transformers import AutoTokenizer
from strands import Agent, tool
from strands_sglang import SGLangClient, SGLangModel
from strands_sglang.tool_parsers import get_tool_parser

@tool
def calculator(expression: str) -> str:
    """Evaluate an arithmetic expression."""
    return str(eval(expression))

async def main():
    model = SGLangModel(
        client=SGLangClient(base_url="http://localhost:30000"),
        tokenizer=AutoTokenizer.from_pretrained("Qwen/Qwen3.5-4B"),
        tool_parser=get_tool_parser("qwen_xml"),
    )
    agent = Agent(model=model, tools=[calculator])
    await agent.invoke_async("What is 25 * 17?")

    # SGLangModel captures the full token trajectory for on-policy RL training
    rollout = model.rollout
    print(rollout.token_ids, rollout.loss_mask, rollout.logprobs)

asyncio.run(main())

RL Training

In principle, Strands-SGLang can be seen as a drop-in agentic rollout service and can be integrated with any RL training framework. A concrete example of training a math coding agent (ReTool) is available at slime/examples/strands_sglang.

Some key highlights of adapting Strands-SGLang to any RL framework:

• Pass tokens and token metadata from Rollout for on-policy rollouts
• Hook your harness with the built-in LoopLimiter (or your own) for controlled rollouts
• Use a shared SGLangClient and HF tokenizer; don't create one instance per rollout
• Classify the rollout's termination reason properly — it shapes reward and sampling

Testing

# Unit tests
pytest tests/unit/ -v

# Integration tests (requires SGLang server)
pytest tests/integration/ -v --sglang-base-url=http://localhost:30000

Contributing

Contributions welcome! Install pre-commit hooks for code style and commit message validation:

uv sync
pre-commit install

This project uses Conventional Commits. Commit messages must follow the format:

<type>(<scope>): <description>

# Examples:
feat(client): add retry backoff configuration
fix(sglang): handle empty response from server
docs: update usage examples

Allowed types: feat, fix, docs, style, refactor, perf, test, build, ci, chore, revert

License

Apache License 2.0 - see LICENSE.