Epsilab Python SDK

Python SDK and CLI for the Epsilab RL Environment Hub.

What is Epsilab?

Epsilab is an open hub for RL environments. Search, run, and export training data from hosted environments, or publish your own with a single command.

• Researchers and teams training models — search and run environments, export GRPO/DPO/SFT/KTO training data, batch evaluation
• Environment builders — publish with epsilab deploy, immutable content-addressed releases, usage analytics
• Open by default — public, shared, unlisted, or private visibility; auto-qualified releases

Installation

pip install epsilab                  # SDK + CLI only
pip install epsilab[training]        # + TRL, Together AI, torch
pip install epsilab[tinker]          # + Tinker (custom training loops)
pip install epsilab[fireworks]       # + Fireworks AI
pip install epsilab[all]             # everything

Quick Start

Deploy an environment

epsilab login
epsilab whoami
epsilab init my-environment
cd my-environment
# edit environment.py and tasks.json
epsilab deploy

The deploy command creates the namespace and listing when needed, builds the image, uploads it, registers immutable task and verifier releases, and makes the environment available for hosted sessions. No UUIDs or registry credentials are required.

Run an environment

epsilab run epsilab/bug-hunter
epsilab run epsilab/bug-hunter \
  --task bug-hunter-easy-train-001 \
  --action 'BUG_LINE: 6
FIX:
return total / len(numbers)'

The first form starts an interactive session. The second performs one action and prints its observation, reward, terminal state, and verification result.

from epsilab import Epsilab

# Credentials are loaded automatically from `epsilab login`
client = Epsilab()

# Browse the hub
listings = client.list_environment_listings(limit=20)
for l in listings:
    print(f"{l.slug:30s}  {l.title}")

# Create a session and interact
session = client.create_environment_session(
    listings[0].deployment_id,
    task_id="bug-hunter-easy-train-001",
    seed=42,
)
session = client.wait_for_session(session)
result = client.environment_step(
    session.session_id,
    {
        "action_type": "submit",
        "content": "BUG_LINE: 6\nFIX:\nreturn total / len(numbers)",
    },
    session_token=session.session_token,
)
print(f"Reward: {result.reward}, Done: {result.done}")

# Export training data
export = client.create_environment_export(
    deployment_id=listings[0].deployment_id,
    format="grpo",
)

Run a long-horizon coding agent

run_agent_episode separates model turns from environment steps. A model may reason for many turns before calling a tool; every request, reasoning chunk, assistant message, tool boundary, result, usage record, cancellation, and error is persisted immediately in the session trace.

from epsilab import AgentToolCall, AgentTurn, AgentUsage, Epsilab

client = Epsilab()

def call_your_model(context):
    # Invoke any provider here. Translate its response into AgentTurn.
    # Do not set a provider token limit if you want an uncapped generation.
    response = your_provider_call(context.history, context.observation)
    return AgentTurn(
        reasoning=response.reasoning,
        message=response.message,
        tool_calls=[
            AgentToolCall(call_id=call.id, name=call.name, arguments=call.arguments)
            for call in response.tool_calls
        ],
        usage=AgentUsage(
            input_tokens=response.input_tokens,
            output_tokens=response.output_tokens,
            cost_usd=response.cost_usd,
        ),
        provider=response.provider,
        model=response.model,
        provider_request_id=response.request_id,
    )

rollout = client.run_agent_episode(
    deployment_id="deployment-id",
    task_id="task-id",
    model_fn=call_your_model,
    max_turns=500,
    cancel_check=my_cancel_event.is_set,
)
print(rollout.stop_reason, rollout.turns_completed, rollout.session.total_reward)

The SDK has no token budget or per-generation cap. max_turns is the only runner limit (maximum 500 model calls); the environment's own immutable task horizon still governs tool actions. See examples/run_long_horizon_agent.py for a provider-neutral adapter skeleton.

Post-training with multiple environments

For generalist RL post-training, train across diverse environments simultaneously — coding, ops, business, etc.:

# Collect data from 3 envs, format as DPO, train locally
python examples/run_environment.py \
    --envs bug-hunter,refactor,test-writer \
    --algorithm dpo \
    --provider local

# Online GRPO across all available envs
python examples/grpo_training.py --envs all --provider tinker --steps 100

The example scripts support any training provider:

Batch evaluation with your policy

Batch creation provisions reproducible sessions; your code still supplies the agent actions. run_batch handles both parts so sessions cannot be left idle:

batch = client.run_batch(
    deployment_id="deployment-id",
    name="regression sweep",
    task_seed_pairs=[
        {"task_id": "task-001", "seed": 42},
        {"task_id": "task-002", "seed": 42},
    ],
    policy_fn=lambda observation, info: my_agent(observation),
)

for session in batch["sessions"]:
    print(session["task_id"], session["total_reward"])

For a batch created in the dashboard or CLI, call client.drive_batch(batch_id, policy_fn=my_policy).

Creating an Environment

An RL environment is a containerized task server. At minimum it needs:

Generated projects use "reward_mode": "continuous", which accepts any finite reward from 0 through 1, including binary rewards. Set it to "binary" when every terminal reward is exactly 0 or 1, or "partial_credit" when fractional rewards represent completed rubric items.

New projects also include a qualification block with a deterministic task and terminal smoke action. epsilab deploy publishes the immutable release first, then queues a hosted compatibility check for that exact release. Publication is not blocked by the check, but hosted sessions become available only after it passes. Inspect progress or retry explicitly:

epsilab env status <release-id>
epsilab env qualify <release-id> --task <task-id> --action '<terminal action>'

For custom action schemas, set qualification.smoke_actions in .epsilab/project.json to one or more valid actions that deterministically end the selected task. A failed check does not hide the published listing.

Environment protocol

The generated server uses the OpenEnv contract and exposes:

GET  /health
POST /reset
POST /step
WS   /ws

/reset accepts a task ID and deterministic seed. /step accepts a typed action and returns an observation, optional reward, and terminal state. Most authors only edit environment.py; the generated server and verifier already implement the transport and replay contracts.

tasks.json format

[
  {
    "task_id": "find-the-bug-001",
    "prompt": "Find and fix the bug in the following code...",
    "difficulty": "easy",
    "split": "train",
    "max_steps": 10
  }
]

Creating an Application Tool

Application tools are reusable plugins (e.g. GitHub, Slack, Calendar) that environments can compose together. A tool needs:

cd my-tool/
epsilab deploy    # auto-detects tool structure

Publishing and Binding Datasets

Datasets are immutable input bundles that an environment release can mount read-only. A dataset targeted at an Application Tool must contain exactly one JSON seed file compatible with that tool release's seed schema.

from epsilab import Epsilab

client = Epsilab()
dataset = client.create_dataset(
    namespace_id="namespace-id",
    slug="crm-history",
    title="CRM History",
    category="business-applications",
    domain="business",
)
artifact = client.upload_dataset_artifact(
    "crm-history.tar.gz",
    manifest={
        "schema_version": 1,
        "schema": {"kind": "appsuite_seed", "version": 1},
        "entries": [{
            "path": "hubspot/seed.json",
            "media_type": "application/json",
            "record_count": 1000,
            "seed_schema_digest": "sha256:" + "0" * 64,  # replace with the tool release value
            "tool_namespace": "epsilab",
            "tool_slug": "hubspot",
        }],
    },
    expanded_size_bytes=123456,
)
client.create_dataset_release(
    dataset_id=dataset.dataset_id,
    release_version="1.0.0",
    license_id="CC-BY-4.0",
    record_count=1000,
    **artifact,
)

Bind the recommended immutable release from an environment project's .epsilab/project.json:

{
  "plugins": ["epsilab/hubspot"],
  "datasets": [{
    "dataset": "epsilab/crm-history",
    "alias": "crm-data",
    "target_tool_alias": "hubspot"
  }]
}

epsilab deploy resolves both references to exact release IDs. At session start the platform verifies the archive digest and declared files, then mounts the bundle at /epsilab/inputs. AppSuite environments replace the targeted tool's initial seed deterministically; untargeted files remain available to environment-specific code.

CLI Commands

All commands support --json for machine-readable output and -v for verbose logging.

Configuration

The SDK resolves credentials in order:

1. Explicit api_key= constructor argument
2. EPSILAB_API_KEY environment variable
3. ~/.epsilab/credentials.json (set by epsilab login)

Most users just need epsilab login — no env vars or code changes required.

Error Handling

from epsilab import Epsilab, AuthError, InsufficientCreditsError, RateLimitError, ApiError

client = Epsilab()

try:
    session = client.create_environment_session("dep-id", task_id="task-001")
except AuthError:
    print("Invalid API key")
except RateLimitError as e:
    print(f"Rate limited. Retry after {e.retry_after}s")
except ApiError as e:
    print(f"API error: {e.status_code}")

The SDK retries automatically on rate limits (429) and transient server errors (500, 502, 503, 504) with exponential backoff and jitter.

Examples

All examples use argparse — run with --help for options. Key flags:

python examples/run_environment.py \
    --envs bug-hunter,refactor,test-writer \
    --algorithm dpo \
    --provider local \
    --sessions-per-env 20

Documentation

License

Apache 2.0 — see LICENSE.