harbor-firecracker 0.1.0

A Harbor BaseEnvironment provider backed by PandaStack Firecracker microVMs — self-hosted evals + fork-based RL rollouts.

harbor-firecracker

The self-hostable Firecracker provider for Harbor — that forks a running agent VM in ~400 ms.
Run agent evals and RL rollouts on your own KVM hosts. Warm a task image once, copy-on-write-branch it into thousands of rollouts that share memory.

Why · Install · Use with Harbor · Fork rollouts · Benchmark

---

Harbor (from the Terminal-Bench team) is becoming the standard harness for agent evaluations and RL environments. It runs each task in an isolated environment behind a pluggable BaseEnvironment interface, and ships providers for Docker, Modal, Daytona, E2B, and Runloop.

There is no open-source, self-hostable Firecracker provider. This is it — backed by PandaStack microVMs.

Why — isolation isn't the point, fork() is

E2B and Daytona already run on Firecracker, so "hardware isolation" is not the differentiator. The differentiator is forking a running VM.

RL and eval workloads do rollouts: take one warmed task environment and run the agent against it N times from the same starting state. Every other provider makes you cold-restore a snapshot per rollout — you pay the full boot + state-load N times. harbor-firecracker warms the task image once, then fork_trees it: each rollout is a copy-on-write branch of the live VM (memory and disk) that diverges in ~400 ms and shares pages with its siblings until it writes.

	Managed providers	harbor-firecracker
Where it runs	their cloud	your KVM hosts
Per rollout	cold-restore a snapshot (linear cost)	CoW-fork a warm VM (~400 ms, shared memory)
50th rollout	as expensive as the 1st	nearly free
Cost model	per-sandbox vendor markup	self-hosted

The 50th rollout being as cheap as the 2nd is the whole pitch.

Install

pip install harbor-firecracker          # the provider + fork-rollout API
pip install "harbor-firecracker[harbor]" # also pulls Harbor, to run `harbor run --env firecracker`
pip install "harbor-firecracker[bench]"  # also pulls matplotlib, for the benchmark chart

Set a PandaStack API key (self-host PandaStack on your KVM hosts, or use the hosted control plane):

export PANDASTACK_API_KEY=pds_...
# export PANDASTACK_API=https://api.pandastack.ai   # default; point at your own control plane to self-host

harbor is an optional peer dependency. import harbor_firecracker works without it; you only need it installed to actually run a Harbor eval.

Use it with Harbor

harbor-firecracker provides a FirecrackerEnvironment whose type() is "firecracker". Register it with Harbor's environment factory, then select it on the command line:

# conftest-style registration, or anywhere before `harbor run`:
from harbor_firecracker import register
register()   # inserts FirecrackerEnvironment into Harbor's environment factory

harbor run -d "<org/benchmark>" -m "<model>" -a "<agent>" --env firecracker

Two integration paths (see register.py for the exact seam):

1. Standalone (this package): call register() to add the provider to Harbor's factory map at runtime.
2. Upstream: vendor FirecrackerEnvironment into harbor/environments/ and add it to Harbor's EnvironmentType enum + factory (a PR into Harbor).

Each Harbor task gets its own microVM. exec() runs the agent's commands inside it (with cwd / env / user / timeout honored), upload_*/download_* move task files and results, and stop() tears it down.

Fork-based RL rollouts

The reason to use this provider. Warm one environment, then fan out:

from harbor_firecracker import fork_rollouts

# Warm a base sandbox (install deps, clone the task, snapshot it), then
# CoW-fork it into 64 rollout environments that share the parent's memory.
with fork_rollouts(base_sandbox, n=64) as pool:
    for rollout in pool:               # each is an isolated, diverging microVM
        rollout.exec("python agent.py ...")
        # collect reward / verifier output, then discard the fork

fork_tree is hard-capped at 16 children per call, so fork_rollouts(n=64) issues batches of 16 under the hood — and going past a single host's memory means scheduling across hosts (see rollouts.py for the batching + the multi-host note). This is an honest constraint, not a "scale to infinity" claim.

Benchmark

The headline graph — fork vs cold-restore on a SWE-bench-Verified-shaped workload — is reproducible, and runs without a live host in simulation:

python -m bench.swebench_rollout --dry-run --n 64 --chart bench/out/chart.png

--dry-run (default) models fork (~400 ms, shared-CoW) vs cold-restore (linear) from measured constants — useful for the chart and CI. --live runs the real workload against your PandaStack hosts (documented in bench/README.md).

License

MIT. Built on PandaStack (Apache-2.0) and targets Harbor (its own license).