OpenReward Python SDK
The official Python SDK for OpenReward — a platform for building, hosting, and training on RL environments for language models.
The SDK has two complementary roles:
- Build environments — define evaluation tasks, expose tools, and serve them via a standards-compliant API that can be deployed on the OpenReward platform.
- Train agents — connect to any environment (local or hosted), run agent loops, and log rollouts with rewards back to OpenReward.
Installation
pip install openreward
For environments that process documents (PDF, DOCX, Excel, PowerPoint):
pip install "openreward[tools]"
Requires Python 3.11+.
Core concepts
Environment
An Environment subclass defines a benchmark or task distribution. Implement three required methods:
| Method | Purpose |
|---|---|
list_splits() |
Return split names, e.g. ["train", "test"] |
list_tasks(split) |
Return a deterministically ordered list of task dicts |
get_prompt() |
Return the task instructions as a list of TextBlock / ImageBlock |
Actions are defined as async methods decorated with @tool. Each tool receives a Pydantic model as input and returns a ToolOutput.
ToolOutput
Every tool returns a ToolOutput containing:
blocks— a list ofTextBlockorImageBlockresultsreward— optional float reward signalfinished— whether the episode is completemetadata— optional arbitrary metadata
Server
Server wraps one or more Environment classes in a FastAPI app and exposes the Open Reward Standard API over HTTP with SSE streaming.
Key endpoints:
| Endpoint | Description |
|---|---|
POST /create |
Spawn a new environment session |
POST /{env}/call |
Execute a tool (streamed via SSE) |
GET /{env}/prompt |
Get the current task prompt |
GET /{env}/tools |
List available tools |
POST /{env}/tasks |
List all tasks for a split |
Sandboxes
Environments that need isolated compute (e.g. code execution) can spin up Docker containers via the sandbox API using SandboxSettings. Containers are managed automatically — started in setup() and torn down in teardown().
Overriding session env vars
When calling a hosted environment, pass env_overrides to Environment.session(...) to point the session at a custom inference backend. Owners of the environment may override any env var; other callers are restricted to OPENAI_BASE_URL, OPENAI_API_KEY, ANTHROPIC_BASE_URL, ANTHROPIC_API_KEY — any other key returns a 400.
env = client.environments.get("GeneralReasoning/counter")
session = env.session(
env_overrides={"OPENAI_BASE_URL": "https://my-vllm.example.com/v1"},
)
Toolsets
Group reusable tools into Toolset classes and compose them across environments via the toolsets class attribute.
Rollout logging
Log agent trajectories with reward signals back to OpenReward for analysis and training. The client's rollout API supports normalized message types as well as raw outputs from Anthropic, OpenAI, and Google GenAI SDKs.
Task difficulty
Environment.get_task_difficulty() returns per-task reward aggregates computed from logged rollouts, grouped by (split, variant, task_index) — useful for spotting which tasks are hardest or have the most variance.
env = client.environments.get("owner/my-env")
for task in env.get_task_difficulty(split="test"):
print(task.task_index, task.avg_reward, task.num_rollouts)
Each result is a TaskDifficulty with task_index, avg_reward, min_reward, max_reward, num_rollouts, split, and variant. Optional filters: split, model_name, min_model_params, max_model_params, training_stage. If the environment was obtained with a variant, results are scoped to that variant (mirroring list_tasks); otherwise rows span all variants. Data is served from a materialized view, so freshness depends on its server-side refresh.
CLI
The orwd CLI helps you scaffold and create environments.
Scaffold a new environment locally
# Minimal environment
orwd init my-env
# Environment with a Docker sandbox for code execution
orwd init my-env --template sandbox
Create an environment on OpenReward
Registers a new environment under your account (requires OPENREWARD_API_KEY):
orwd create my-env --description "A short description of my environment"
By default the environment is created under your personal namespace. To create it under an organisation you are a member of, pass --namespace:
orwd create my-env --description "A short description" --namespace my-org
Pass --private to make the environment private:
orwd create my-env --description "A short description" --private
Deploying to OpenReward
- Push your environment to a GitHub repository.
- Connect the repository in the OpenReward dashboard.
- Configure compute resources (CPU, memory, scaling).
- Every push to the connected branch triggers an automatic build and deployment.
Your environment is then accessible to any agent via the OpenReward API using the username/environment-name namespace.
Environment variables
| Variable | Description |
|---|---|
OPENREWARD_API_KEY |
API key for authentication |
OPENREWARD_URL |
Override base URL (default: https://openreward.ai) |
OPENREWARD_USE_STRUCTURED_LOGS |
Set to 1 for JSON logging (recommended in production) |
OPENREWARD_ROLLOUT_LOGGING_FORMAT |
pretty or structured for rollout log output |
Documentation
Full documentation, guides, and examples are at docs.openreward.ai.
License
Apache 2.0
3 maintainers
Release files for openreward 0.1.159
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| openreward-0.1.159.tar.gz | 273.4 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| openreward-0.1.159-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 500.6 kB
Release files / openreward-0.1.159.tar.gz
| Download URL | openreward-0.1.159.tar.gz |
|---|---|
| Size | 273.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
dfbfa829eb56185313e4d3465c589e17dafff741b2928411048caba0f058a92f
|
|
BLAKE2b-256 checksum How to use checksums |
47edbda25b7edb69739d6fbee498f1e9b3eed87e40959ec4b93fb3e8eac00ae2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.10.6 {"installer":{"name":"uv","version":"0.10.6","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / openreward-0.1.159-py3-none-any.whl
| Download URL | openreward-0.1.159-py3-none-any.whl |
|---|---|
| Size | 227.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
0a0248a53b0dc6750872faa3a6f7a5560cfba7ac483443f8ece0e8c4d4670be3
|
|
BLAKE2b-256 checksum How to use checksums |
a8f21ae4e92fe4b09cac492827143cd6c22225036a86550154a5a844c54cf6d0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.10.6 {"installer":{"name":"uv","version":"0.10.6","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
