k8s-agent-sandbox-awx

A broker-backed, kubeconfig-free overlay for the Agentic Sandbox client. It adds a BrokerSandboxClient that drives the sandbox control plane (create → wait ready → terminate, commands, and files) through one identity broker service over HTTPS instead of the Kubernetes API or a public router. The broker proxies authorized data operations to its internal router, so client applications need no kubeconfig and no direct Kubernetes or router access.

pip install k8s-agent-sandbox-awx==0.9.0

from k8s_agent_sandbox_awx import Sandbox

sandbox = Sandbox.create(keepalive_ttl_seconds=0)
result = sandbox.commands.run('echo "Hello from Agent Sandbox!"')
print(result.stdout)
# Leave it available until its timeout; call sandbox.kill() for early cleanup.

Streaming and background commands

SDK 0.6.0+ supports managed commands on Python runtime v0.3.0 and Codex runtime v0.2.0, through broker v0.11.0. Create a new sandbox after the runtime upgrade; existing claimed sandboxes are not replaced. Plain run() calls fall back to the old execution endpoint on older runtimes, but the new options require the updated runtime and broker.

SDK 0.7.0 makes command strings shell scripts, like E2B. Add callbacks to stream output while run() waits for the final result:

from k8s_agent_sandbox_awx import Sandbox

sandbox = Sandbox.create()
result = sandbox.commands.run(
    'echo hello; sleep 1; echo world',
    on_stdout=lambda data: print(data, end="", flush=True),
    on_stderr=lambda data: print(data, end="", flush=True),
)
print(result)

Stream ten numbered progress messages, one per second:

result = sandbox.commands.run(
    'for i in {1..10}; do echo "Progress $i/10"; sleep 1; done',
    on_stdout=lambda data: print(data, end="", flush=True),
)
print(result.exit_code)

Callbacks receive text chunks, not necessarily complete lines. Use unbuffered output (python3 -u or flush=True) in the program when you need prompt output. The result still contains the collected stdout, stderr, and exit code. Callbacks are optional; without them run() simply returns the result after completion.

Start a background command, wait later, or cancel its process group:

task = sandbox.commands.run(["sleep", "60"], background=True, timeout=120)
print(task.command_id)
print(task.status()["state"])

# Reattach to a known command using the same sandbox/caller:
task = sandbox.commands.get(task.command_id)
# result = task.wait(timeout=10, on_stdout=print)  # wait timeout does NOT kill it
result = task.kill()  # cancel this command, not the sandbox; safe to repeat
print(result.state)   # cancelled

Optional stdin is available only for background commands:

task = sandbox.commands.run(["cat"], background=True, stdin=True)
task.send_stdin("hello\n", eof=True)
print(task.wait().stdout)

timeout limits command execution (default 60 seconds), capped by the runtime's SANDBOX_EXEC_TIMEOUT_SECONDS (default 300 seconds). task.wait(timeout=...) only limits waiting; call kill() to cancel. Sandbox expiry still kills all work; background commands do not extend its lifetime automatically. Cancellation kills the process group, including normal descendants; it is not a separate security boundary against processes deliberately escaping their group.

Environment overrides are per-command. cwd is an absolute path inside the sandbox; choose the template's PVC path for durable outputs.

Shell behavior and migration from 0.6.x

In 0.7.0, strings run through /bin/bash -c: chains (;, &&), pipes, redirects, variables and glob expansion work directly, with or without callbacks. This is a change from 0.6.x, which parsed strings with shlex and executed the resulting argv directly. Use shell=False to preserve that behavior:

sandbox.commands.run("echo '$HOME'", shell=False)  # literal $HOME
sandbox.commands.run(["echo", user_input])  # literal argument, no shell
sandbox.commands.run('echo "$DEMO"', envs={"DEMO": "hello"}, cwd="/app")

Do not interpolate untrusted values into shell strings. Prefer argv lists for programmatic arguments. Lists always execute directly; shell=True with a list is rejected. Bash must be present in custom templates (the standard Python and Codex templates include it). We do not use a login shell, so envs and cwd are preserved. Plain calls on older runtimes also preserve the selected shell behavior via the legacy endpoint, but streaming/background options require the managed-command runtime.

The runtime retains up to 1 MiB of combined output per command and reports output_truncated=True when it reaches the byte/event limit. It drains further output to avoid blocking the process. It allows 16 running managed commands and retains at most 64 handles. Results expire after 15 minutes, or earlier under retention pressure. These are runtime safeguards, not per-caller sandbox quotas. Handles/results are in-memory and do not survive runtime restart or sandbox deletion. They are not part of the PostgreSQL audit store.

Output streams reconnect with a cursor every 20 seconds through the broker, refreshing the caller token and rechecking ownership and sandbox.exec policy. Disconnecting does not cancel background commands. Do not concurrently call wait() on the same handle. A fresh handle replays its retained output.

Filesystem and workspace

SDK 0.8.0 adds scoped file APIs through broker v0.12.0, Python runtime v0.4.0 and Codex runtime v0.3.0. Create a new sandbox after upgrading; existing claimed sandboxes retain their old runtime. These APIs do not fall back to old endpoints. Existing SDK 0.7 /app operations still work with the new runtimes.

sandbox.files.make_dir("/workspace/project")
sandbox.files.write("/workspace/project/hello.txt", "Hello workspace!")
print(sandbox.files.read("/workspace/project/hello.txt", format="text"))
print(sandbox.files.list("/workspace/project"))
sandbox.files.move("/workspace/project/hello.txt", "/workspace/project/result.txt")
print(sandbox.files.get_info("/workspace/project/result.txt"))
print(sandbox.files.exists("/workspace/project/result.txt"))
sandbox.files.remove("/workspace/project", recursive=True)

/workspace is the per-sandbox PVC in both standard templates; /app/workspace aliases the same volume. Files survive container restart while that PVC exists, not sandbox deletion by default. Export important results before timeout. This is not cross-sandbox storage or a memory snapshot. Relative paths stay relative to /app, which is container-local outside its workspace mount. File APIs reject other roots, parent traversal and symlink traversal; commands still run as the container user and are not constrained by the file API. Symlinks can be listed or removed, but not read through.

read() returns bytes by default; format="text" decodes UTF-8. Writes accept bytes or strings, atomically replace regular files and create missing parents. Read/write limits are 64 MiB per file. Listing is bounded to 2,000 direct children. FileInfo includes name, path, type, size, Unix mod_time, modified_ns and numeric POSIX permissions. Move stays within one root and never overwrites a destination. Removing a nonempty directory requires recursive=True; a missing path returns False. File mutations are not automatically retried.

# Run an editor/build command separately while this iterator is active.
watch = sandbox.files.watch_dir("/workspace/project", timeout=30)
try:
    for event in watch:
        print(event.type, event.path)  # created / modified / removed
finally:
    watch.close()  # Stop early if desired.

Watching starts on iteration, polls direct children every 500 ms, and is best-effort: fast transient changes can coalesce or be missed. It is not recursive, inotify or an audit history. Up to 16 active watches are allowed per runtime. Streams reconnect every 20 seconds with fresh ownership/policy checks and diff the next baseline. Timeout is finite and directory removal raises an error. It does not extend sandbox TTL unless activity keepalive was explicitly enabled. Every operation still uses broker authentication, current claim ownership, OPA read/write permission and audit logging.

How it works

The client presents an audience-bound Kubernetes or GitLab OIDC token to the broker. The broker cryptographically verifies the caller from that token, creates the SandboxClaim under its own identity, and stamps the verified caller identity onto the claim — attribution the client cannot forge.

Set both broker_url and api_url to the broker URL. The model rejects a different api_url to prevent accidental direct-router bypass.

This package is a purely additive overlay: it builds on k8s-agent-sandbox-base (a seamed re-publish of the upstream client) and edits zero upstream files. The import package is k8s_agent_sandbox_awx.

Configuration

The zero-argument API reads:

Timeouts

Sandbox.create() defaults to 300 seconds (five minutes). An explicit shutdown_after_seconds overrides AGENT_SANDBOX_TTL_SECONDS (or legacy SANDBOX_TTL_SECONDS), which overrides that default. Values must be positive. Commands do not reset the deadline unless keepalive is explicitly enabled. sandbox.set_timeout(600) resets it to ten minutes from the call and can shorten or extend it. The broker enforces its hard lifetime from the original claim creation time (24 hours in US nonprod); extensions cannot reset that cap, and caller policy may impose a shorter per-request maximum.

Plain assignment leaves the sandbox running after the script exits. with Sandbox.create() as sandbox: and sandbox.kill() intentionally delete it immediately. Job ttlSecondsAfterFinished cleans the caller Job/Pod, not the sandbox. A replacement caller Pod cannot take ownership of the previous Pod's claim merely by using the same ServiceAccount.

These defaults apply to SDK 0.5.2. When using older 0.5.0, the broker supplies the five-minute default, but pass keepalive_ttl_seconds=0 explicitly for fixed-deadline behavior.

Kubernetes normally needs no Python authentication code:

from k8s_agent_sandbox_awx import Sandbox

with Sandbox.create() as sandbox:
    print(sandbox.commands.run("python3 -c 'print(6 * 7)'").stdout)

Mount a projected ServiceAccount token at the configured path with its audience set to the broker URL. The file is re-read on every request so token rotation is honored.

Cleanup activity

If sandbox creation or readiness fails, upstream attempts a cleanup delete. The broker overlay marks that delete with request_purpose: create_cleanup; normal explicit deletes are not marked. This is diagnostic client intent only and never bypasses admission, action-policy or ownership checks. Broker v0.10.2 and the console can distinguish create, delete and SDK-reported cleanup even when caller admission fails. Older records cannot be retroactively labeled. SDK 0.5.2 includes this cleanup attribution; older 0.5.0 does not.

GitLab supplies the same zero-argument API using a native ID token:

job:
  id_tokens:
    AGENT_SANDBOX_TOKEN:
      aud: https://sandbox-broker.example
  variables:
    AGENT_SANDBOX_BROKER_URL: https://sandbox-broker.example
  script:
    - python app.py

Explicit authentication objects are available when environment discovery is not appropriate:

from k8s_agent_sandbox_awx import KubernetesAuth, OIDCTokenAuth, Sandbox

k8s = Sandbox.create(auth=KubernetesAuth("/var/run/secrets/tokens/broker"))
ci = Sandbox.create(auth=OIDCTokenAuth.from_env("MY_ID_TOKEN"))

The SDK deliberately has no provider="gitlab" or provider="kubernetes" switch. It transports the token; the broker derives the provider from the verified issuer and applies the matching policy.

Local Google / gcloud user access

With broker v0.13.0+, operators can explicitly configure an OIDC caller's audience override. An omitted override still requires the broker audience. All callers from one issuer must agree on that audience. Kubernetes and GitLab keep their existing checks; audience validation is never skipped.

For approved personal Google accounts:

gcloud auth login
export AGENT_SANDBOX_BROKER_URL=https://sandbox-broker.ai.gcp-us.dev.awx.im
export AGENT_SANDBOX_TOKEN="$(gcloud auth print-identity-token)"
python app.py
unset AGENT_SANDBOX_TOKEN

Use SDK 0.8.0 and Sandbox.create() as usual; no kubeconfig, projected token mount or Kubernetes API access is required. This uses an ID token, not print-access-token. The SDK does not run gcloud or automatically refresh this token; re-export when it expires (normally one hour). Keep it out of logs, shell traces, Git and YAML.

Google CLI user tokens use issuer https://accounts.google.com and the shared Cloud SDK audience 32555940559.apps.googleusercontent.com. This is not a sandbox-specific audience, so this mode is an explicit developer convenience, not the preferred long-lived workload identity. Operators must require exact verified-email allowlists, optionally the hd domain and stable numeric subject, and corresponding OPA grants. Each user owns sandboxes by their signed Google subject, not their email. Only selected verified claims are forwarded to policy. Google email and hd must be scalar strings and email_verified a true boolean.

The US nonprod policy permits elvis.cai@airwallex.com, jungrae.kim@airwallex.com and pradithya.pura@airwallex.com, in domain airwallex.com, with lifecycle, commands and files on Python/Codex pools, a one-hour timeout cap, and no tool scopes. Other users remain denied. Console Okta roles are independent. devops@airwallex.com group membership is not checked; a future Directory API integration needs Workspace-admin authorization. Google tokens contain no group membership claim automatically. Service-account impersonation is a separate option with the broker audience; all impersonators then share that account's ownership.

Reference: https://github.com/agent-substrate/substrate/blob/main/docs/authentication.md Unlike its no-RBAC example, this broker still applies admission, OPA and ownership.

GKE checkpoints (0.9.0)

Use an approved snapshot pool and your normal broker authentication:

sandbox = Sandbox.create(warmpool="python-snapshot-warmpool", shutdown_after_seconds=900)
try:
    saved = sandbox.snapshots.create()
    sandbox.suspend(snapshot_id=saved.snapshot_id)
    restored = sandbox.resume()
    assert restored.restored_from_snapshot
finally:
    sandbox.kill()

Also supports sandbox.restore(snapshot_id), snapshots.list(), snapshots.get(snapshot_id), snapshots.delete(snapshot_id), and snapshots.lifecycle(). Operations require explicit broker/OPA grants and return typed results or raise exceptions. They do not reset the shutdown deadline. Restore uses the same live claim, not a new sandbox. Eight checkpoints per claim maximum; claim kill/expiry cleans up checkpoint storage. Client timeouts do not cancel operations. GKE preview: same-node rootfs and memory recovery; PVC/remote service consistency and cross-node recovery are not guaranteed. Snapshots can contain credentials; protect storage accordingly.

Requirements

• A deployed identity broker service reachable over HTTPS.
• An OIDC token whose audience matches the broker's. Kubernetes projected tokens and GitLab id_tokens: are supported by the broker policy.
• k8s-agent-sandbox-base (installed automatically as a dependency).

License

Apache-2.0 — see LICENSE.