lab-kit
lab-kit turns your coding agent into the staff of a research lab: it pre-registers each experiment, locks the plan before the first number, runs the lab's own code against the lock, scores the outcome against it, and records only numbers that re-derive from committed evidence. It is built on folio. folio decides what a document is; lab-kit decides when a result counts.
You do not run lab-kit yourself, and you do not walk the agent through each step. You install it by handing your agent one line, set up the lab with its question, and then plan each mission with the agent in plain words. Once you approve the plan, the agent runs it end to end: it drafts the protocol, has a reviewer read it, locks it, runs it, scores it, has the numbers re-derived by an independent reviewer, records the result and writes the report, running the gate before every commit. It stops only where the plan and the method say it must.
Install
Paste this into your coding agent, in the repository that will hold the lab:
Install lab-kit here: run `uv tool install lab-kit-cli --with-executables-from folio-kb` (or `pipx install --include-deps lab-kit-cli`), then `lab-kit init`, then read .agents/skills/set-up-lab/SKILL.md and follow it.
Installing lab-kit brings folio with it; the flag puts folio's command on the path too. The agent asks you at most three questions in one message (the lab's question, where the library lives, which paths are frozen), sets up the library with folio's lab pack, records the question as Q-1, and leaves the gate passing. A longer version of the prompt is in SETUP.md. lab-kit needs Python 3.10 or later.
Give it a mission
1. Plan the mission together. Once the lab is set up, state an objective:
Mission: find out whether our pi estimator's error falls as 1/sqrt(n).
The plan-mission skill reads the lab, then asks in one message only what it cannot look up: "How many sample sizes and repeats count as enough? Token-free only? What would make you stop it early?" It drafts the plan: the question served, observable milestones, the scope, the spend, what it never touches, and where it must stop. You change what you want and say "approved". It records your approval and starts nothing.
2. Let it run. The run-mission skill carries out the approved plan alone; it refuses an unapproved one. It dispatches workers through the experiment and review skills, sends a reviewer before the protocol locks and before any result counts, and keeps ops/STATE.md current so a crashed session resumes from the record. It comes back to you only at the plan's stops, or when a run would spend past the cap, a change would touch a frozen surface, a lock or a recorded result, or the work would leave the plan's scope.
It reports the result by id, the misses as plainly as the hits, and its recommended next step. Meanwhile, ask "how is the mission going?" or "what should we do next?".
The loop
The documents (questions, protocols, results, claims, reports, the journal) are folio's, through its lab pack. The steps between them are lab-kit's skills: set-up-lab (once), plan-mission (with you, until you approve), run-mission (alone: dispatch, supervise, recover, report), experiment (draft, lock, launch, watch) and review (fold, score, record, report). Four agent roles do the work: scout, runner, reviewer and reporter.
What each step rules out
The method is pre-registration, as clinical trials use it, applied to an agent: the hypothesis, the predictions that could refute it and the rules that decide them are fixed before the first run. Each step closes off a known way for research to go wrong, and a check holds it.
| Step | Rules out | Held by |
|---|---|---|
| Predictions, each with a confidence, frozen before the run | Fitting the hypothesis to the result after seeing it | lab-lock-intact, lab-run-after-lock |
| Decision rules, one of them a kill rule | Choosing the threshold after the data; an idea that cannot fail | lab-score-exact |
| One variable, every other setting pinned | Tuning other settings until something works | lab-roster-frozen |
| Run output sealed by a manifest | Quiet edits and silent re-runs | lab-evidence-sealed |
| A re-derive command for every result | Numbers typed in, rounded or invented | lab-rederive |
| An independent reviewer pass | Analysis that does not support the claim | lab-result-grounded |
| Misses and kills recorded like hits | Writing up only what worked | lab-scored-reported (a warning) and the journal |
The checks
The gate, lab-kit check, runs folio's checks and then the lab's. It runs offline, names every problem in one pass, and changes nothing. CI runs the same command.
- Locks. A locked protocol has a lock record, and its bytes still match it (
lab-lock-recorded,lab-lock-intact). - Runs. Every run started after its lock, used exactly the pinned configuration, and its evidence matches its manifest (
lab-run-after-lock,lab-roster-frozen,lab-evidence-sealed). - Results. Every live result re-derives exactly, rests on a finished run of a locked protocol, and passed an independent reviewer (
lab-rederive,lab-result-grounded). - Scores. A scorecard scores exactly the ids the lock names (
lab-score-exact), and a scored experiment is reported or explained (lab-scored-reported). - Lab files. Ids resolve, frozen surfaces are untouched, scripts pass
--selftest, nothing runs under an unapproved mission, spend stays within its cap and its standing budget, records only grow, the method is current, and the state file stays a short pointer.
Together they hold a chain from every number on a page back to the lock:
The gate checks that the record is consistent. Whether it is honest is the reviewer's job.
An example
examples/monte-carlo-lab asks one question: does the error of a Monte Carlo estimate of pi shrink as 1/sqrt(n)? Its protocol, locked before the run, draws 100 seeded estimates at each of five sample sizes from 64 to 16,384 points, in pure Python, in under a second. The RMS error fell with a fitted log-log slope of -0.493 (R-2), so the hypothesis is kept; two of the four predictions missed, and the report says so as plainly as it says the rest. R-2 supersedes R-1, which fitted the wrong error measure. Every record in it was made by the loop above, and lab-kit check passes on it with no error and no warning.
Open it with your agent and ask "check this lab" or "re-derive R-2".
Reference
The lab-kit command is the interface for agents and CI, the way git is: init, check, experiment, lock, run, runs, score, rederive, freeze, status and version. Each is specified in docs/spec/lab-model.md §6, the contract the skills, the roles, the checks and the command all build on. To work on lab-kit itself, see CONTRIBUTING.md. Changes: CHANGELOG.md.
Related
folio, giro and lab-kit are three kits for working with coding agents: folio keeps the documents, giro takes a spec to proof on a branch, and lab-kit runs research on top of folio.
Made by Pier Mallozzi.
License
MIT. See LICENSE.
Metadata
Release files for lab-kit-cli 0.2.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| lab_kit_cli-0.2.0.tar.gz | 140.9 kB | With provenance | Details for lab_kit_cli-0.2.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| lab_kit_cli-0.2.0-py3-none-any.whl | Python 3 | none | any | Details for lab_kit_cli-0.2.0-py3-none-any.whl |
Total release size: 211.0 kB
Release files / lab_kit_cli-0.2.0.tar.gz
| Download URL | lab_kit_cli-0.2.0.tar.gz |
|---|---|
| Size | 140.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
371ed7727c9a54453a99e04acd36939a9c5a711e65f17ed837ed2fc11da0f7b3
|
|
BLAKE2b-256 checksum How to use checksums |
21a2338df759e4dcfc63ebbdb999505d2b7fcb83e1967e4c24b818e0acc3d3d5
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 29, 2026.
Transparency logRelease files / lab_kit_cli-0.2.0-py3-none-any.whl
| Download URL | lab_kit_cli-0.2.0-py3-none-any.whl |
|---|---|
| Size | 70.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
a2b5918617bf0f49ade6688b1bb8a182ed255f8792e5182f563f0d4919482907
|
|
BLAKE2b-256 checksum How to use checksums |
0ce0d14bdbf9dc1e6befe76523538f13be5391ba74497eb5d9ba7f56bf020842
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 29, 2026.
Transparency logSecurity for lab-kit-cli 0.2.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for lab_kit_cli-0.2.0.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 29, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.