pydantic-ai-exedev

Run a Pydantic AI agent's workspace on an exe.dev VM, so the native FileSystem, Shell and other Pydantic AI Harness capabilities work there. You can also manage those VMs from your app, outside any agent run.

This is a community package, not part of Pydantic AI. It is 0.x: the API may change between minor releases.

Install

uv add pydantic-ai-exedev "pydantic-ai-harness[anthropic]"

Put an exe.dev API token in EXE_TOKEN. Scope it to a tag, so it can only manage the VMs you tag:

ssh exe.dev ssh-key generate-api-key --tag=agents --exp=30d

In an agent

from pydantic_ai import Agent
from pydantic_ai_exedev import ExeDevSandbox, VMSpec
from pydantic_ai_harness import FileSystem, Shell

sandbox = ExeDevSandbox(spec=VMSpec(image='ubuntu:24.04', working_dir='app'), scope_tag='agents')
agent = Agent('anthropic:claude-opus-5-5', capabilities=[sandbox, FileSystem(), Shell()])

result = agent.run_sync('Clone https://github.com/pydantic/pydantic-ai and run the harness tests.')

# Later, on the same VM: pass its ref, or continue from the message history.
agent.run_sync('Fix the first failing test.', workspace=result.workspace.ref)

The first workspace operation of a run with no ref provisions a VM from spec. A run given a WorkspaceRef, by workspace= or from message history, attaches to that VM. Pass workspace='new' for a fresh one. Ending a run never deletes the VM.

Provision first, then hand the ref to a workspace

When your app owns the VM's lifecycle, provision it with ExeDevVMs, keep its WorkspaceRef, and attach a backend to it:

from pydantic_ai import Agent
from pydantic_ai_exedev import ExeDevSandboxBackend, ExeDevVMs, VMSpec
from pydantic_ai_harness import FileSystem, Shell

spec = VMSpec(name='build-42', tags=['req-7f3a'], image='ubuntu:24.04', working_dir='app')

async with ExeDevVMs(scope_tag='agents') as vms:
    state = await vms.provision(spec)  # creates the VM, waits until it runs commands, makes `app`

backend = ExeDevSandboxBackend(ref=state.ref, spec=spec, scope_tag='agents')
try:
    agent = Agent('anthropic:claude-opus-5-5', capabilities=[FileSystem(), Shell()])
    await agent.run('Set up the project.', workspace=backend)
finally:
    await backend.aclose()  # closes the client it made; the VM keeps running

ExeDevSandboxBackend(ref=...) attaches without creating, resuming or changing the VM. spec then only says where commands start (working_dir), which must already exist. Without ref, the backend provisions from spec on first use, exactly as ExeDevVMs.provision does: the capability, the backend and ExeDevVMs share one implementation.

VMSpec: image or base VM, resources, network

A VMSpec describes a new VM made from an image, or a copy of a base VM:

VMSpec(
    name='build-42',  # optional: exe.dev generates one
    comment='nightly build',  # at most 200 bytes
    tags=['req-7f3a', 'ci'],
    image='ubuntu:24.04',  # or base='golden', a VM to copy
    cpu=4,
    memory='8G',
    disk='50G',
    egress='integrations',  # reach only exe.dev integrations, not the internet ('all')
    standalone=False,  # reserved capacity (needs a plan with it)
    integrations=['github'],  # new VMs only
    working_dir='app',  # created while getting the VM ready
)

It is checked when built, with no I/O. A combination exe.dev can't apply raises ValueError; nothing is silently dropped:

A copy's egress defaults to its base's. The base is looked up first, through the same scope check as any VM you name.

Scope tag

scope_tag (on ExeDevVMs, ExeDevSandboxBackend and ExeDevSandbox) is added to every VM they create. Every operation on a VM you name refuses one without it, with VMOutOfScopeError: attach, inspect, pause, resume, initialize, destroy, and a copy's base. It keeps a ref from message history, or a typo, away from your other VMs.

It is not authorization. Anyone whose token can reach a VM can still reach it outside this package. Use an API token scoped to the same tag to enforce it. A copy gets the scope tag only by inheriting it from its base, so copy_base_tags=False with a scope_tag is refused.

When a create's result is unknown

A create or copy request can fail after it reached exe.dev: a network error, an unreadable response, or a server error (5xx, including the 504 after /exec's 30-second limit). The VM may or may not exist. provision then raises ProvisionOutcomeUnknownError and retries nothing. A backend that hit one raises it again on every later operation rather than create a second VM.

To recover, give the spec a stable name and a tag unique to the request, and look for it:

from pydantic_ai_exedev import ProvisionOutcomeUnknownError

spec = VMSpec(name='build-42', tags=['req-7f3a'])
try:
    state = await vms.provision(spec)
except ProvisionOutcomeUnknownError:
    found = await vms.lookup(spec)
    if found.status == 'found':
        state = found.vm  # it was created: use it (initialize it, below, if needed)
    elif found.status == 'not_found':
        ...  # not visible now: look again later, or provision again
    else:  # 'conflict' or 'ambiguous'
        ...  # a VM you can't confirm is yours: inspect it yourself

lookup never creates, resumes or changes a VM. It needs tags, because a name alone could belong to a VM you made some other way.

A stable name also means a repeated provision is refused when the name is taken, rather than making a second VM. This is not exactly-once creation: a name or tags make a VM findable, nothing more. A 4xx from create is a definite refusal (such as a name in use) and propagates as the SDK's exedev.APIError. A cancelled provision leaves no VM, because the SDK deletes one it made before re-raising; if that delete fails, lookup finds the VM.

When getting a VM ready fails

Once the VM exists, provision waits until it runs commands. For a copy, it then sets the spec's tags and comment, and finally makes working_dir. If any of that fails, it raises VMInitializationError, whose ref names the VM. The VM is kept, never deleted for you. Retry with await vms.initialize(error.ref, spec), which is safe to repeat and changes nothing else, or delete it. A backend that provisioned the VM records the ref first and retries initialization on its next operation.

Lifecycle outside a run

async with ExeDevVMs(scope_tag='agents') as vms:
    state = await vms.inspect(ref)  # VMState: status, tags, comment, egress, region, created_at, ...
    await vms.pause(ref)  # saves memory to disk, frees CPU and memory
    await vms.resume(ref)  # waits until it runs commands
    vm = await vms.get_vm(ref)  # the SDK's exedev.AsyncVM, for anything not wrapped here
    deleted = await vms.destroy(ref)  # True if this call deleted it, False if it didn't exist

Each takes a WorkspaceRef or a VM name. Only resume resumes a paused VM: inspect, lookup, pause, get_vm and destroy never do. A workspace command on a paused VM resumes it first: exe.dev refuses the command before it starts, so it runs once, after the resume. ExeDevSandbox.vms() gives a manager with the capability's client and scope tag, and ExeDevSandbox.destroy(ref) deletes directly.

Permissions the token needs: new (provision), cp (copy), ls (lookup, inspect, attach, and before pause, resume and destroy), pause, resume, rm (destroy), tag and comment (a copy's tags and comment), and ssh (commands, readiness waits, working_dir, and the flush before a pause or copy).

Clients, retries and errors

Clients. Pass client=exedev.AsyncClient(...) to ExeDevVMs, ExeDevSandboxBackend or ExeDevSandbox, and it is never closed for you; that is also how you set the SDK's retry policy (max_retries, or your own http_client). Without one, a client is made from EXE_TOKEN on first use:

• ExeDevVMs closes it in aclose() or at the end of async with, including on errors and cancellation.
• A backend closes it in aclose(), after waiting for a VM it is provisioning, so that VM's ref is recorded first.
• ExeDevSandbox closes the client of the backend it built for a run when that run ends, however it ends. It never closes a workspace that another run handed in (a parent's, given to a subagent).
• result.workspace still works after the run: its next operation makes a new client, which await result.workspace.backend.aclose() closes.

aclose() never touches the VM: it doesn't delete, pause or recreate it.

Cancellation during cleanup. Once aclose() (or the end of an async with block, or of a run) starts closing a client, the close finishes before a cancellation reaches the caller. This holds for asyncio's Task.cancel(), however many times it is called, and for AnyIO's cancel scopes. The cancellation is not lost, only delayed: it is raised as soon as the close is done, and no cleanup is left running in the background. There is no timeout, so a close that hangs holds the cancellation back for as long as it hangs.

If closing fails, the client is kept, and calling aclose() again retries. Without a cancellation, the close's error is raised as is. If the caller was cancelled meanwhile, the CancelledError is raised, with the close's error as its __cause__.

Retries. This package adds no retries of its own. It makes one exception: a command on a paused VM is refused before it starts, so it is run once more after the resume. A command whose result is unknown is never run again.

Errors. All of these are WorkspaceErrors, which durable engines don't retry:

Other exedev errors, such as a network failure before anything ran, a rate limit after the client's retries, or a server error on a safe operation, propagate unchanged, so a durable engine can retry them. Messages and reprs name VMs and tags, never tokens, environment values or response bodies.

Commands, files, and Shell's background work

Commands run in the VM user's login shell, starting in working_dir. The environment is the VM's login environment, plus the capability's or backend's env, plus each call's env (including Shell(env=...)), with later layers winning. An argv list is passed literally; a string needs shell=True, as the WorkspaceBackend protocol says.

Files go through Workspace's shell fallback, in 64 KiB pieces, so binary files of any size round-trip, at about one request per 64 KiB. For large trees, use tar through Shell instead.

Output the command leaves open. exedev's run returns once the command has exited and its output has closed. A background process that inherits stdout or stderr (server &) keeps the command waiting until it exits or the timeout stops everything. Redirect it: server > server.log 2>&1 &.

Shell's background tools. Shell's start_command, check_command and stop_command, and its persistent shell tool, need nothing from this package. Their launcher detaches the job into its own session with setsid (exeuntu has it) and returns, so the job keeps running after the launcher's command ends, after the run, and across backends. A later run, attached by ref with a new backend, checks or stops the job by its ID or PID. The job's logs and status live in the VM, under .pydantic-ai-harness/ in the working directory.

Cancellation and timeouts. Cancelling a command, or its timeout, closes the request. exe.dev then stops the command's process group, but not a job that detached into its own session. A deleted VM fails a running command with VMNotFoundError. A dropped connection to a live VM, or a response the SDK can't read, raises CommandOutcomeUnknownError; the command is never re-run.

Output limits. A single command's output over 8 MiB raises WorkspaceOutputLimitError. Shell's own max_output_chars keeps the tail of what the model sees.

What's verified, and what isn't

The test suite runs against a local fake of the exedev SDK. The fake follows the SDK's documented contract (exedev 0.4.0) and runs commands as real local processes. The checks include core's WorkspaceBackendSuite, Harness's FileSystem and Shell across runs and backends, and every provisioning, recovery and lifecycle path above.

These are assumptions from the SDK's source and documentation that only real exe.dev can confirm. uv run pytest -m live checks most of them, but those tests have not been run yet:

• A deleted VM fails a running command, and a paused one refuses a command before it starts.
• ls and pause don't resume a VM.
• run waits for output a background process inherits.
• A detached Shell job outlives its launcher on a real VM.
• A copy request takes the documented flags.
• The status exe.dev returns for a VM name that is taken. The fake uses 422; the SDK doesn't document it.
• Whether a create that timed out at the gateway (504) went on to make the VM.

Development

uv sync
uv run pytest -m "not live"
uv run ruff check . && uv run ruff format --check . && uv run ty check
EXE_TOKEN=... uv run pytest -m live   # creates billed VMs, tagged pydantic-ai-exedev-test, and deletes them

Releasing

Releases are published to PyPI by .github/workflows/release.yml, using Trusted Publishing. PRs and pushes to main run CI; publishing a GitHub Release runs CI again, builds and checks the distributions, then uploads them to PyPI. Pushing a tag alone does not publish the package.

One-time setup:

• Add a PyPI pending publisher for project pydantic-ai-exedev, owner daikeren, repository pydantic-ai-exedev, workflow release.yml, and environment pypi.
• Create the repository's pypi environment under Settings > Environments.

To release:

1. Update project.version in pyproject.toml, run uv lock, and commit the release changes.

2. Push the commit and wait for CI to pass.

3. Create a GitHub Release with a tag matching the version, such as v0.1.0, pointing to that commit.

4. Check the release notes and click Publish release. This triggers the PyPI upload.

5. Wait for the Publish to PyPI workflow to succeed, then verify the published package:

   uv run --isolated --no-project --with pydantic-ai-exedev==0.1.0 python -I -c 'import pydantic_ai_exedev'
   

Use the released version in the verification command. Each PyPI version is immutable; corrections need a new version.

License

MIT.