Skip to main content

harbor-firecracker

The self-hostable Firecracker provider for Harbor — that forks a running agent VM in ~400 ms.
Run agent evals and RL rollouts on your own KVM hosts. Warm a task image once, copy-on-write-branch it into thousands of rollouts that share memory.

Why · Install · Use with Harbor · Fork rollouts · Benchmark


Harbor (from the Terminal-Bench team) is becoming the standard harness for agent evaluations and RL environments. It runs each task in an isolated environment behind a pluggable BaseEnvironment interface, and ships providers for Docker, Modal, Daytona, E2B, and Runloop.

There is no open-source, self-hostable Firecracker provider. This is it — backed by PandaStack microVMs.

Why — isolation isn't the point, fork() is

E2B and Daytona already run on Firecracker, so "hardware isolation" is not the differentiator. The differentiator is forking a running VM.

RL and eval workloads do rollouts: take one warmed task environment and run the agent against it N times from the same starting state. Every other provider makes you cold-restore a snapshot per rollout — you pay the full boot + state-load N times. harbor-firecracker warms the task image once, then fork_trees it: each rollout is a copy-on-write branch of the live VM (memory and disk) that diverges in ~400 ms and shares pages with its siblings until it writes.

Managed providers harbor-firecracker
Where it runs their cloud your KVM hosts
Per rollout cold-restore a snapshot (linear cost) CoW-fork a warm VM (~400 ms, shared memory)
50th rollout as expensive as the 1st nearly free
Cost model per-sandbox vendor markup self-hosted

The 50th rollout being as cheap as the 2nd is the whole pitch.

Install

pip install harbor-firecracker          # the provider + fork-rollout API
pip install "harbor-firecracker[harbor]" # also pulls Harbor, to run `harbor run --env firecracker`
pip install "harbor-firecracker[bench]"  # also pulls matplotlib, for the benchmark chart

Set a PandaStack API key (self-host PandaStack on your KVM hosts, or use the hosted control plane):

export PANDASTACK_API_KEY=pds_...
# export PANDASTACK_API=https://api.pandastack.ai   # default; point at your own control plane to self-host

harbor is an optional peer dependency. import harbor_firecracker works without it; you only need it installed to actually run a Harbor eval.

Use it with Harbor

harbor-firecracker provides a FirecrackerEnvironment whose type() is "firecracker". Register it with Harbor's environment factory, then select it on the command line:

# conftest-style registration, or anywhere before `harbor run`:
from harbor_firecracker import register
register()   # inserts FirecrackerEnvironment into Harbor's environment factory
harbor run -d "<org/benchmark>" -m "<model>" -a "<agent>" --env firecracker

Two integration paths (see register.py for the exact seam):

  1. Standalone (this package): call register() to add the provider to Harbor's factory map at runtime.
  2. Upstream: vendor FirecrackerEnvironment into harbor/environments/ and add it to Harbor's EnvironmentType enum + factory (a PR into Harbor).

Each Harbor task gets its own microVM. exec() runs the agent's commands inside it (with cwd / env / user / timeout honored), upload_*/download_* move task files and results, and stop() tears it down.

Fork-based RL rollouts

The reason to use this provider. Warm one environment, then fan out:

from harbor_firecracker import fork_rollouts

# Warm a base sandbox (install deps, clone the task, snapshot it), then
# CoW-fork it into 64 rollout environments that share the parent's memory.
with fork_rollouts(base_sandbox, n=64) as pool:
    for rollout in pool:               # each is an isolated, diverging microVM
        rollout.exec("python agent.py ...")
        # collect reward / verifier output, then discard the fork

fork_tree is hard-capped at 16 children per call, so fork_rollouts(n=64) issues batches of 16 under the hood — and going past a single host's memory means scheduling across hosts (see rollouts.py for the batching + the multi-host note). This is an honest constraint, not a "scale to infinity" claim.

Benchmark

The headline graph — fork vs cold-restore on a SWE-bench-Verified-shaped workload — is reproducible, and runs without a live host in simulation:

python -m bench.swebench_rollout --dry-run --n 64 --chart bench/out/chart.png

--dry-run (default) models fork (~400 ms, shared-CoW) vs cold-restore (linear) from measured constants — useful for the chart and CI. --live runs the real workload against your PandaStack hosts (documented in bench/README.md).

License

MIT. Built on PandaStack (Apache-2.0) and targets Harbor (its own license).

Metadata

Release files for harbor-firecracker 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for harbor-firecracker 0.1.0
File Size Uploaded Provenance Actions
harbor_firecracker-0.1.0.tar.gz 37.3 kB Without provenance Details for harbor_firecracker-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for harbor-firecracker 0.1.0
File Interpreter ABI Platform Actions
harbor_firecracker-0.1.0-py3-none-any.whl Python 3 none any Details for harbor_firecracker-0.1.0-py3-none-any.whl

Total release size: 61.2 kB

Release files / harbor_firecracker-0.1.0.tar.gz

Download URL harbor_firecracker-0.1.0.tar.gz
Size 37.3 kB
Tags Source
SHA-256 checksum
How to use checksums
c0e5f61b52e2b24abeb9b4153705f2f7aede19baf86cc5c398d2e8ff266e0e21
BLAKE2b-256 checksum
How to use checksums
63efd0e76e2ebc4501fd6d857e284801669ba5e157d14fbb51c39fda02c33351
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.12.8

Release files / harbor_firecracker-0.1.0-py3-none-any.whl

Download URL harbor_firecracker-0.1.0-py3-none-any.whl
Size 23.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
ead12e1de5482e41e14bc768974e44aefde051ff35e722facb1ad819976a0489
BLAKE2b-256 checksum
How to use checksums
52d762e39da5acfb33fcb56fe51b18fb42c52bbb55300ef0f41542460789256f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.12.8

Security for harbor-firecracker 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page