modal-computer-use

modal-computer-use turns a Modal Sandbox into a remotely controllable Linux desktop through a typed, provider-neutral Python SDK and an in-Sandbox daemon.

This is an independent project using Modal.

Quick start

Use Python 3.12 or later and uv. Install the Modal extra from PyPI:

uv add "modal-computer-use[modal]"

The Modal extra supports the Modal 1.5 line and requires Modal 1.5.3 or later.

Save this as quickstart.py. Choose one supported narrow Modal region selector for both the Function and the Sandbox. The resource values are application choices, not SDK defaults.

import uuid

import modal

from modal_computer_use import AsyncComputerSandbox, ComputerConfig, ComputerSessionHandle

APP_NAME = "computer-use-quickstart"
REGION = "us-west"

app = modal.App(APP_NAME)
function_image = modal.Image.debian_slim(python_version="3.12").pip_install(
    "modal-computer-use[modal]"
)


@app.function(
    image=function_image,
    region=REGION,
    cpu=1.0,
    memory=2048,
    min_containers=0,
    retries=0,
    timeout=900,
)
async def trajectory(handle: ComputerSessionHandle, run_id: str) -> tuple[int, int]:
    async with handle.borrow_async(run_id=run_id, function_region=REGION) as computer:
        result = await computer.step(
            [
                {"type": "move", "x": 320, "y": 240},
                {"type": "click", "x": 320, "y": 240},
            ],
            continue_on_error=False,
        )
        return result.screenshot.width, result.screenshot.height


@app.local_entrypoint()
async def main() -> None:
    config = ComputerConfig(
        runtime={"modal_environment": "main", "modal_region": REGION},
        resources={"profile": "browser", "cpu": 1.0, "memory_mib": 2048},
        browser={"kind": "chromium"},
    )
    async with AsyncComputerSandbox.create(config=config, app_name=APP_NAME) as owner:
        size = await trajectory.remote.aio(
            owner.session_handle(), f"quickstart_{uuid.uuid4().hex}"
        )
        print(*size)

Run it:

uv run modal run --env main quickstart.py

The async owner creates one desktop and produces a versioned handle. An application-owned Modal Function enters one borrow for the whole trajectory and reuses one pooled async HTTP client. computer.step() sends the ordered action batch and returns its immediate, byte-backed screenshot in one request. The Function releases the lease before the owner terminates the Sandbox. This lifecycle is the optimized default topology. Warm capacity stays off because min_containers=0; enable paid idle capacity only after measuring the tradeoff.

Core API

AsyncComputerSandbox plus ComputerSessionHandle.borrow_async() is the primary Modal trajectory interface. The provider model loop belongs in the application-owned Modal Function. Enclose the repeated loop in one borrow, then call computer.step() for each ordered action array and its immediate post-action frame.

The synchronous SDK, direct daemon clients, attach flows, REST routes, and idempotency tools remain available as low-level compatibility surfaces. Use them for local control, direct-daemon work, debugging, recovery, or an application that owns its own lifecycle. They do not establish the article-backed placed topology by themselves.

Action batches validate the full request before execution and stop after the first failure unless the caller enables continuation. The trailing-screenshot option is a retained low-level capability. The default model-loop path is the borrowed computer.step() interface, which returns action results, an immediate screenshot, and timing metadata.

See the API reference for namespace semantics and the generated OpenAPI schema for HTTP request and response shapes.

Lifecycle and limits

The owner, placed Function, and borrow have separate cleanup scopes. Native async provisioning supports cancellation and cleanup without reducing cold allocation or desktop startup. Measure allocation, Function dispatch, borrow entry, and repeated warm operations separately.

The daemon reserves the cost of a complete ordered batch before its first mutation. The default is the portable baseline for the minimum tested Sandbox: 100 normalized tokens per second with a 400-token burst. Measure another setup before raising both values. See Performance for placement, capacity, and measurement guidance.

Performance

The figure shows July 2026 p50 latency for six warm operations, based on 30 successful samples per cell. Lower is better. A separate 100-sample benchmark measured one click followed by the next full screenshot. computer.step() measured 43.13 ms p50 and 46.35 ms p95. See the action-to-frame report for the provider paths, timer boundary, screenshot formats, and configuration limits.

The benchmark results give p95 values and explain how each path was configured and measured.

Examples

Documentation

Local development

See the local development guide for daemon startup, mock and X11 backends, synchronous and async clients, authentication, and repository checks.

Security

The daemon can control the desktop and access clipboard contents, screenshots, recordings, and artifacts. Do not expose it without authentication.

See the security policy for reporting vulnerabilities and the runtime security guide for deployment guidance.