Thunder Sandbox for Python

Create short-lived GPU sandboxes, run commands over SSH, and move files with a small, typed Python API.

Thunder Sandbox uses the same account and credentials as the Thunder CLI. It handles sandbox lifecycle, SSH key creation, waiting for readiness, command execution, uploads, and downloads through one synchronous and asynchronous Python API.

Installation

Thunder Sandbox requires Python 3.10 or newer.

pip install thunder-sandbox

To install the current development branch directly from GitHub:

pip install git+https://github.com/Thunder-Compute/thunder-sandbox.git

Authenticate with the Thunder CLI before using the library:

tnr login

Alternatively, set TNR_API_TOKEN and, when using a non-default API endpoint, TNR_API_URL. API endpoints must use HTTPS.

Quick start

import thunder_sandbox as thunder

sandbox = thunder.Sandbox.create(
    cpu=4,
    memory=32,
    storage=50,
    gpu_type=thunder.GPUType.A6000,
    gpu_count=1,
    timeout=900,
)

try:
    sandbox.wait_until_ready()

    process = sandbox.exec("nvidia-smi")
    stdout = process.stdout.read()
    exit_code = process.wait()
    if exit_code != 0:
        raise RuntimeError(process.stderr.read())
    print(stdout)
finally:
    sandbox.terminate()

wait_until_ready() asks Thunder to hold the request open until the sandbox is ready, so it returns moments after startup finishes rather than on the next poll. Its timeout is enforced on the client: each held request is bounded here and retried until the sandbox is ready, fails, or the timeout passes. An API without this endpoint is polled instead.

Sandboxes are addressed by id. Sandbox.create() generates an Ed25519 key pair and stores it under ~/.thunder/sandbox_keys/<sandbox-id>. The public key is immutable for the lifetime of the sandbox.

A name is an optional label. It must be free of any other live sandbox in the organization, and it is released once a sandbox finishes, so the same label can be reused later. A name never addresses a sandbox:

sandbox = thunder.Sandbox.create(name="training-run", gpu_type=thunder.GPUType.H100)
print(sandbox.id, sandbox.name)

# Claiming a name a live sandbox already holds raises ConflictError.
# Looking one up searches live sandboxes; prefer Sandbox.from_id.
same = thunder.Sandbox.from_name("training-run")

Handling errors

Conditions worth retrying are typed, so they can be caught without matching on message text. Each carries the API's code, the HTTP status, and the server's retry_after hint when one was sent:

try:
    sandbox = thunder.Sandbox.create(gpu_type=thunder.GPUType.H100)
except thunder.CapacityError as exc:
    # No free GPU of that type right now; the request was fine.
    time.sleep(exc.retry_after or 30)
except thunder.RetryableError:
    # Rate limited, or Thunder could not service the request.
    ...

Run commands

Pass command arguments separately to avoid local shell interpretation:

process = sandbox.exec("python3", "-c", "print('hello from Thunder')")
print(process.stdout.read())
exit_code = process.wait()

Non-PTY commands are launched as detached durable jobs. Their execution, status, and output files do not depend on the SSH channel which submitted them. If the connection is lost, the SDK reconnects and resumes observing the same job rather than launching it again. Save process.id to recover it later:

process_id = process.id
recovered = sandbox.get_process(process_id)
exit_code = recovered.wait()

Durable commands do not accept stdin; stdin operations raise io.UnsupportedOperation. Pass input through arguments, environment variables, or uploaded files. Commands created with pty=True retain interactive stdin but remain attached to their SSH channel. process.is_durable reports which mode a handle uses.

PTY commands are intentionally never replayed or recovered: input and terminal state cannot be reconstructed safely after their SSH channel disappears. A PTY disconnect raises ConnectionError and leaves the remote command's final state unknown. Use the default pty=False mode for unattended or long-running work.

Durable stdout and stderr are reconnectable streams. The SDK reads short SFTP chunks by byte offset and advances its cursor only after a complete chunk is in client memory, so a lost SSH connection resumes without gaps or duplicates. Text mode also preserves UTF-8 characters split across chunks.

Output is captured by default. Long-running commands which do not need one or both streams can redirect them directly to /dev/null in the durable launcher:

process = sandbox.exec(
    "python3", "train.py", stdout="discard", stderr="capture"
)

Once a job is terminal and both captured streams have reached EOF, its remote job directory is removed automatically. Pass retain=True to keep it available for later get_process() recovery, and call process.cleanup() when finished. Explicit cleanup is idempotent and waits for a running job to finish.

process.terminate() is also durable. It records termination intent, signals the entire remote process group, and reconnects safely if the SSH acknowledgement is lost. Jobs which do not exit after a five-second SIGTERM grace period are stopped with SIGKILL; wait() then returns 143 or 137.

Commands can set a working directory, environment variables, a timeout, or a pseudo-terminal:

process = sandbox.exec(
    "python3",
    "train.py",
    workdir="/home/ubuntu/project",
    env={"MODEL": "llama", "DEBUG": "1"},
    timeout=600,
)

Transfer files

sandbox.upload("model.py", "/home/ubuntu/model.py")
sandbox.upload("dataset", "/home/ubuntu/dataset", recursive=True)

sandbox.download("/home/ubuntu/results.json", "results.json")
sandbox.download("/home/ubuntu/checkpoints", "checkpoints", recursive=True)

Transfers intentionally restart from the beginning rather than maintaining a resumable byte manifest. Uploads first write to an isolated remote staging path; downloads first write beside the local destination. If SSH disconnects, the SDK reconnects and repeats the complete staged transfer. Completed files are published with an atomic rename, so partial data is never presented as the destination. Directory merges begin only after the network transfer completes.

Network policies

Sandboxes have unrestricted outbound access by default. Restriction is always explicit:

# No outbound internet access.
closed = thunder.Sandbox.create(block_network=True)

# Only the specified CIDRs and domains are permitted.
restricted = thunder.Sandbox.create(
    outbound_cidr_allowlist=["203.0.113.0/24"],
    outbound_domain_allowlist=["pypi.org", "files.pythonhosted.org"],
)

CIDR and domain allowlists are independent. Supply both when restricted workloads need both direct IP and DNS-based access.

For policy updates, None leaves that dimension unrestricted, while an empty sequence blocks it. Each call replaces the complete policy rather than merging with the previous allowlists.

Replace the complete outbound policy of a running sandbox with the same options used at creation:

# Permit package downloads while blocking other destinations.
restricted.update_network_policy(
    outbound_domain_allowlist=["pypi.org", "files.pythonhosted.org"],
)

# Block all outbound network access.
restricted.update_network_policy(block_network=True)

# Restore unrestricted outbound access.
restricted.update_network_policy()

update_network_policy() returns after Thunder accepts the desired policy; enforcement on the sandbox's node converges asynchronously. Tightening a policy blocks new connections but does not currently guarantee that already-established connections are terminated.

Environment and lifetime

sandbox = thunder.Sandbox.create(
    env={"EXPERIMENT": "baseline"},
    timeout=3600,
)

timeout is the sandbox lifetime in seconds. Set it to None to create a sandbox without an enforced TTL.

Work with existing sandboxes

with thunder.Client.from_cli() as client:
    for sandbox in client.list_sandboxes():
        print(sandbox.id, sandbox.status.value)

    sandbox = client.get_sandbox("sbx-0123456789abcdef")
    sandbox.wait_until_ready(timeout=300)
    print(" ".join(sandbox.ssh_command))

The private SSH key must still exist locally to execute commands or transfer files against an existing sandbox.

Async API

Every blocking operation has an awaitable _async twin on the same public class. This makes it possible to use one import and pass Client, Sandbox, and Process objects between synchronous and asynchronous application code:

import asyncio
import thunder_sandbox as thunder


async def main() -> None:
    sandbox = await thunder.Sandbox.create_async(
        gpu_type=thunder.GPUType.A6000,
        gpu_count=1,
    )
    try:
        await sandbox.wait_until_ready_async()
        process = await sandbox.exec_async("nvidia-smi")
        exit_code = await process.wait_async()
        if exit_code != 0:
            raise RuntimeError(await process.stderr.read_async())
        print(await process.stdout.read_async())
    finally:
        await sandbox.terminate_async()


asyncio.run(main())

An image-backed sandbox ignores the image's ENTRYPOINT and CMD and keeps the container alive for the sandbox lifetime. Commands run inside that container through sandbox.exec(...):

sandbox = thunder.Sandbox.create(image=thunder.Image.from_registry("ubuntu:24.04"))
process = sandbox.exec("sh", "-c", "echo hello")

Positional arguments to Sandbox.create start the first process through the same exec path after the sandbox becomes ready. Without an image, exec runs directly in the guest VM.

Configuration

Configuration is resolved from the following sources:

1. Explicit ClientConfig values.
2. TNR_API_TOKEN and TNR_API_URL environment variables.
3. Thunder CLI state in ~/.thunder/cli_config.json.
4. The default Thunder API endpoint.

Set TNR_HOME to use a different directory for CLI state and sandbox SSH keys.

License

Thunder Sandbox is available under the Apache License 2.0.