inspect-briq

Run Inspect AI sandboxes on briq. Each sample gets its own microVM, destroyed when the sample ends.

from inspect_ai import Task, task
from inspect_ai.dataset import Sample
from inspect_ai.scorer import includes
from inspect_ai.solver import generate, use_tools
from inspect_ai.tool import bash
from inspect_ai.util import SandboxEnvironmentSpec
from inspect_briq import BriqSandboxEnvironmentConfig


@task
def kernel():
    return Task(
        dataset=[
            Sample(
                input="Run `uname -s` with the bash tool and tell me its output.",
                target="Linux",
                files={"notes.txt": "one file per sample, copied in before the first message"},
            )
        ],
        solver=[use_tools(bash(timeout=30)), generate()],
        scorer=includes(),
        sandbox=SandboxEnvironmentSpec("briq", BriqSandboxEnvironmentConfig(size="medium")),
    )

pip install inspect-briq
export BRIQ_API_KEY=...
inspect eval kernel.py --model anthropic/claude-sonnet-4-5

sandbox="briq" uses the defaults. Python 3.11 or later (what Inspect needs), asyncio backend.

Configure

BriqSandboxEnvironmentConfig, or a JSON file with the same fields: sandbox=("briq", "briq.json").

The API key is read from BRIQ_API_KEY only: a key in the configuration would end up in the eval log.

BriqSandboxEnvironmentConfig(
    size="large",
    env={"MODE": "test"},
    secret_env=["GITHUB_TOKEN"],  # read from os.environ, never stored in the log
)

secret_env fails with unsupported when the API cannot keep a value write-only. It never falls back to a plain variable.

What is supported

timeout_retry and concurrency have no effect: a command that outlives its timeout is stopped and raises TimeoutError at once. Limit parallel sandboxes with --max-sandboxes.

Errors:

A command whose end is not known raises BriqError with code="outcome_unknown". It is never repeated.

Clean up

A sample's sandbox is destroyed when the sample ends, also when it fails or is cancelled. If the evaluation is killed, its sandboxes stay: they sleep after sleep and are not billed while asleep. Destroy them from the dashboard, or with briq_cloud: AsyncBriq().sandbox(id).destroy().

Develop

Needs uv.

uv sync
uv run pytest          # offline, the briq client is faked
uv run ruff check . && uv run ruff format --check .
uv run mypy

License

MIT