GL Computer Use

Description

A typed Python SDK for desktop automation via natural-language prompts. GL Computer Use wraps cloud desktop sandboxes and computer-use agents into a clean async API with live streaming, human-in-the-loop takeover, structured observability, and swappable providers.

Key Features

• Streaming and non-streaming run modes: run() for live events, run_once() for a single result, run_sync() for non-async scripts and Jupyter notebooks.
• Swappable agents: cua (trycua/cua, default) or agents (simular-ai/Agent-S).
• Swappable sandboxes: e2b (E2B Desktop, default) or opensandbox (Alibaba OpenSandbox).
• Live desktop URL: noVNC streaming URL surfaced via the SANDBOX_READY event or StreamClient.stream_url.
• Human-in-the-loop takeover: pause an agent loop and hand control to a human, then resume with optional guidance.
• Artifact storage: local disk by default, MinIO/S3 via the minio extra.
• Structured logging with optional OpenTelemetry tracing/metrics and Sentry via the observability extra.
• Custom provider registration: plug in your own sandbox, agent, or artifact store without modifying the SDK.

---

Installation

Install the core SDK:

pip install gl-computer-use

Install optional extras only when you need them:

pip install "gl-computer-use[recording]"     # WebM session recording via Playwright
pip install "gl-computer-use[agents]"        # Agent-S (simular-ai) support
pip install "gl-computer-use[opensandbox]"   # Alibaba OpenSandbox support
pip install "gl-computer-use[minio]"         # MinIO / S3-compatible artifact store
pip install "gl-computer-use[observability]" # OTLP tracing/metrics + Sentry via gl-observability
pip install "gl-computer-use[all]"           # all of the above

API keys required at runtime:

1. E2B API key — e2b.dev (when using sandbox="e2b")
2. Anthropic API key (for the default claude-sonnet-4-6 model) or OpenAI API key

Session recording setup (optional, one-time)

WebM recordings require Playwright's Chromium binaries (~130 MB, stored under ~/.cache/ms-playwright/):

pip install "gl-computer-use[recording]"
gl-computer-use-setup

If you skip this step, the SDK falls back to GIF recording via screenshot stitching.

---

Quick Start

Streaming events

run() returns a StreamClient; iterate it to receive events. The terminal TASK_COMPLETED event carries the final TaskResult.

import asyncio
from gl_computer_use import GLComputerUseClient


async def main() -> None:
    client = GLComputerUseClient()
    stream = await client.run("Open Firefox and navigate to google.com")

    async for event in stream:
        if event.event_type == "SANDBOX_READY" and event.stream_url:
            print(f"Watch live at: {event.stream_url}")
        elif event.event_type == "STEP_COMPLETED":
            print(f"Step {event.step_index}: {event.action.type if event.action else '—'}")
        elif event.event_type == "TASK_COMPLETED":
            print(f"Status: {event.result.status}")
            print(f"Output: {event.result.output}")


asyncio.run(main())

Fire-and-forget async

run_once() returns a TaskResult directly when the task finishes. Raises TaskFailedError / TaskCancelledError on non-COMPLETED outcomes.

import asyncio
from gl_computer_use import GLComputerUseClient


async def main() -> None:
    client = GLComputerUseClient()
    result = await client.run_once("Open a terminal and check Python version")
    print(result.status, result.output, len(result.steps))


asyncio.run(main())

Synchronous / Jupyter

run_sync() is a plain synchronous method — no asyncio.run(), no await. It detects whether an event loop is already running and dispatches via ThreadPoolExecutor when needed, so it works in regular scripts and Jupyter notebooks (no nest_asyncio required).

from gl_computer_use import GLComputerUseClient

result = GLComputerUseClient().run_sync("Open the file manager")
print(result.status)

---

Configuration

Configuration is read from environment variables (prefix GLCU_) or by passing a GLComputerUseConfig object directly. Create a .env file in your working directory:

GLCU_E2B_API_KEY=sk-e2b-...
GLCU_ANTHROPIC_API_KEY=sk-ant-...

# Optional overrides
GLCU_MODEL=anthropic/claude-sonnet-4-6
GLCU_TASK_TIMEOUT=300
GLCU_MAX_STEPS=50

Critical fields:

Variable	Default	Description
GLCU_E2B_API_KEY	None	E2B Desktop API key (required when sandbox="e2b")
GLCU_ANTHROPIC_API_KEY	None	Anthropic API key (required for anthropic/* models)
GLCU_OPENAI_API_KEY	None	OpenAI API key (required for openai/* models)
GLCU_MODEL	"anthropic/claude-sonnet-4-6"	LLM in provider/name format
GLCU_AGENT	"cua"	Agent provider: "cua" or "agents"
GLCU_SANDBOX	"e2b"	Sandbox provider: "e2b" or "opensandbox"
GLCU_ARTIFACT	"local"	Artifact store: "local" or "minio"
GLCU_TASK_TIMEOUT	600.0	How long one run may take, in seconds
GLCU_SANDBOX_TIMEOUT	600	Sandbox lifetime in seconds, measured from create (E2B clamps to 3600) — keep it above GLCU_TASK_TIMEOUT so provisioning and bring-up do not eat the task's budget
GLCU_SANDBOX_IMAGE	""	Image/template to boot; empty means the backend default (desktop for E2B, public.ecr.aws/c1z4u5m0/external/desktop-sandbox:latest for OpenSandbox)
GLCU_SANDBOX_PROVISION_TIMEOUT	300	HTTP timeout for create/resume calls, and the budget for polling a snapshot to Ready — raise it when the server is slow to answer
GLCU_SANDBOX_READY_TIMEOUT	30	OpenSandbox health-check timeout after create/restore; an unready sandbox is killed when this expires — raise it when cold image pulls are slow
GLCU_SANDBOX_REQUEST_TIMEOUT	60	HTTP timeout for actions and screenshots once the sandbox is up
GLCU_MAX_STEPS	100	Maximum agent loop iterations; also the Agent-S ceiling unless GLCU_AGENTS_MAX_STEPS is set
GLCU_LOCAL_ARTIFACT_DIR	"./artifacts"	Directory for saved screenshots and recordings
GLCU_LOG_LEVEL	"INFO"	DEBUG, INFO, WARNING, or ERROR
GLCU_LOG_FORMAT	"json"	"json" (structured) or "console" (human-readable)
GLCU_DEFAULT_DISPOSITION	"destroy"	Teardown when no disposition is passed: "destroy" or "snapshot" (see Snapshot & Resume)
GLCU_STRICT_SNAPSHOT	False	Turn best-effort snapshot/resume failures into hard ConfigError/SnapshotError
GLCU_TRAJECTORY_MAX_IMAGES	None	Cap retained screenshots in replayed history (unbounded by default; 5 recommended)
GLCU_TRAJECTORY_MAX_BYTES	None	Hard ceiling (bytes) on persisted trajectory JSON
GLCU_KEEP_SNAPSHOT_HISTORY	False	Retain every snapshot instead of rolling-GC'ing predecessors (E2B)
GLCU_ALLOW_MODEL_DRIFT	False	Allow resuming a token whose model differs from the configured model

OpenSandbox, MinIO, Agent-S, and observability (OTLP/Sentry/PII) have additional GLCU_* env vars — see GLComputerUseConfig in gl_computer_use/config.py for the full list.

Timeouts

The five timeout knobs nest, and three of them are named *_SANDBOX_*, so it is easy to reach for the wrong one:

Knob	Bounds	Raise it when
GLCU_SANDBOX_TIMEOUT	Total sandbox lifetime (wall clock, from create)	Long tasks die mid-run as transport errors
GLCU_TASK_TIMEOUT	One run (wall clock)	The agent legitimately needs more steps
GLCU_SANDBOX_PROVISION_TIMEOUT	A single HTTP request while creating/resuming; also the snapshot-to-Ready polling budget	The server is slow to answer
GLCU_SANDBOX_READY_TIMEOUT	The OpenSandbox health check, after which the SDK kills the sandbox	Cold image pulls are slow
GLCU_SANDBOX_REQUEST_TIMEOUT	A single HTTP request once the desktop is up	Actions or screenshots time out on a healthy box

PROVISION_TIMEOUT and REQUEST_TIMEOUT bound one request each, not a phase — a phase issuing twenty calls can far outlast either. Only SANDBOX_TIMEOUT and TASK_TIMEOUT bound elapsed time.

READY_TIMEOUT is the one to size against your own server: run a task on an uncached node once and check how long provisioning takes before the health check passes.

---

Provider Agnosticism

Swap agents and sandboxes via config alone — no code changes:

Agent	Sandbox	Config
CUA (default)	E2B (default)	GLComputerUseClient()
CUA	OpenSandbox	GLComputerUseConfig(sandbox="opensandbox")
Agent-S	E2B	GLComputerUseConfig(agent="agents")
Agent-S	OpenSandbox	GLComputerUseConfig(agent="agents", sandbox="opensandbox")

from gl_computer_use import GLComputerUseClient, GLComputerUseConfig

client = GLComputerUseClient(GLComputerUseConfig(agent="agents", sandbox="opensandbox"))

---

Runtime API

The client exposes three run methods:

Method	Returns	Use when
await client.run(prompt, ...)	StreamClient	You need live event streaming or the SANDBOX_READY URL before the task finishes
await client.run_once(prompt, ...)	TaskResult	You only need the final result, async context
client.run_sync(prompt, ...)	TaskResult	You only need the final result, non-async script or Jupyter notebook

All three methods accept the same parameters:

Parameter	Type	Default	Description
prompt	str	—	Task description
config	GLComputerUseConfig | None	None	Per-call config override
timeout	float | None	None	Max seconds (falls back to config.timeout)
files	list[File] | None	None	Files to upload to the sandbox before the task
retrieve_files	list[str] | None	None	Sandbox paths to download after completion
on_takeover_needed	Callable | None	None	Takeover callback

run_once() and run_sync() raise TaskFailedError / TaskCancelledError directly instead of returning a result with a non-COMPLETED status.

---

Live Desktop (noVNC)

When using the E2B sandbox, a noVNC HTTP endpoint is started alongside the desktop. The SDK waits until that endpoint is reachable before surfacing the URL.

# Option A — pre-iteration attribute
stream = await client.run("do something")
print(stream.stream_url)

# Option B — first SANDBOX_READY event
async for event in stream:
    if event.event_type == "SANDBOX_READY" and event.stream_url:
        webbrowser.open(event.stream_url)

---

Takeover

Pass on_takeover_needed to run() / run_once() / run_sync(). The agent pauses when a takeover condition is detected, and your callback receives a TakeoverContext with the session state and a resume() function. Without a callback, a TakeoverRequiredError is raised. See examples/takeover.py and examples/takeover_caller_initiated.py.

---

Snapshot & Resume

A session can be paused — its sandbox state and the agent's conversation trajectory are persisted — and later resumed from a ResumeToken. This is fully backward-compatible: the teardown disposition defaults to destroy, so existing callers are unaffected.

Pass disposition="snapshot" to capture a token, then pass it back via resume_from:

client = GLComputerUseClient()

# 1. Run and snapshot instead of destroying the sandbox.
result = await client.run_once("Open Firefox and log into the dashboard", disposition="snapshot")
token = result.resume_token            # a ResumeToken; token.to_json() to persist it

# 2. Later — resume from where it left off.
result = await client.run_once("Now download this month's report", resume_from=token)

resume_from accepts a ResumeToken, its dict, or its JSON-string form. Set GLCU_DEFAULT_DISPOSITION=snapshot to snapshot by default without passing the argument each call.

Artifact store requirement

Snapshot history is written through the artifact store, so the store must support history persistence. The built-in local and minio stores do; a custom store must set the class attribute supports_history = True and implement save_history / load_history / delete_history.

If the store does not support history, the snapshot still captures the desktop but the token is emitted with history_ref=None — the restored sandbox has no agent memory. By default this is logged at ERROR and the run continues. Set strict_snapshot=True (GLCU_STRICT_SNAPSHOT=true) to turn it — and any history-persistence or snapshot-capture failure — into a hard ConfigError / SnapshotError instead.

Trajectory growth

Every resume re-feeds the full trajectory to the model, and it grows without bound by default. To keep cost and payload size in check:

• trajectory_max_images (GLCU_TRAJECTORY_MAX_IMAGES, default None = unbounded) — caps the number of most-recent screenshots retained in the replayed history. A small value such as 5 is recommended for long or repeatedly-resumed threads.
• trajectory_max_bytes (GLCU_TRAJECTORY_MAX_BYTES, default None) — a hard ceiling on the persisted trajectory JSON. When exceeded, the snapshot path fails before writing (raising SnapshotError under strict_snapshot, otherwise skipping history).

Provider semantics

• E2B takes true copy-on-write snapshots; predecessor snapshots are garbage-collected as a thread advances (set keep_snapshot_history=True to retain every snapshot for branching/forking).
• OpenSandbox pauses and resumes the same container — delete_snapshot is a no-op there. Paused threads accumulate and remain billable until explicitly released, so release sessions you no longer intend to resume.

Resuming a token whose model differs from the configured model raises ResumeError, because trajectory/image/caching formats differ across providers. Set allow_model_drift=true (GLCU_ALLOW_MODEL_DRIFT=true) to bypass that check at your own risk; a compatibility warning is logged.

---

Errors

All SDK exceptions extend GLComputerUseError:

• ConfigError — bad or missing credentials.
• SandboxProvisionError — the sandbox could not be allocated.
• GLTimeoutError — no event received within the configured timeout.
• TaskFailedError — the agent terminated with an error (TASK_FAILED).
• TaskCancelledError — the task was cancelled (TASK_CANCELLED).
• TakeoverRequiredError — takeover was needed but no callback was supplied.
• SnapshotError — a snapshot/history-persistence failure (only raised when strict_snapshot=True; see Snapshot & Resume).
• ResumeError — a token could not be resumed (e.g. model drift without allow_model_drift).

from gl_computer_use import (
    GLComputerUseClient,
    ConfigError,
    SandboxProvisionError,
    GLTimeoutError,
    TaskFailedError,
)

try:
    result = await GLComputerUseClient().run_once("do something", timeout=60.0)
except ConfigError as e:
    print("Check your API keys:", e)
except SandboxProvisionError as e:
    print("Sandbox failed to start:", e)
except GLTimeoutError as e:
    print("Took too long:", e)
except TaskFailedError as e:
    print("Agent failed:", e)

---

Observability

The SDK uses structlog for structured logging (JSON by default; set GLCU_LOG_FORMAT=console for human-readable output). Every line carries session_id, task_id, and component. Distributed tracing and metrics via OTLP, plus Sentry error tracking, are available through the observability extra and delegated to GDP Labs' gl-observability SDK. Optional regex-based PII redaction is enabled with GLCU_PII_REDACTION_ENABLED=true.

---

Custom Providers

Plug in alternative sandboxes, agents, or artifact stores without modifying the SDK:

from gl_computer_use import register_sandbox, GLComputerUseClient, GLComputerUseConfig
from gl_computer_use.sandbox.base import BaseSandbox


class MyCustomSandbox(BaseSandbox):
    ...  # implement abstract methods


register_sandbox("my-sandbox", MyCustomSandbox)
client = GLComputerUseClient(config=GLComputerUseConfig(sandbox="my-sandbox"))

register_agent and register_artifact work the same way for custom agents and artifact stores.

---

Local Development Setup

git clone git@github.com:GDP-ADMIN/gl-sdk.git
cd gl-sdk/libs/gl-computer-use
uv sync --all-extras
uv run gl-computer-use-setup
source .venv/bin/activate

Run checks:

uv run pytest           # tests
uv run ruff check .     # lint
uv run ruff check --fix # auto-fix lint
uv run mypy gl_computer_use/  # type-check

---

Contributing

Please refer to the Python Style Guide for code style, documentation standards, and SCA requirements.