Gadfly
Gadfly tests whether a decision rests on the evidence you think it rests on. It removes or changes one declared line under fixed interventions, has a scorer, the reader under test, answer each trial, and reads one verdict by fixed rules, so the scorer answers and never judges. The method is When meaning is load-bearing.
- Fixed rules, not a model: a verdict follows from the kept answers, so anyone can check it.
- It says what it does not know: an unresolved verdict names the check still open, kept apart from the evidence.
- Kept records: every answer is tied to its exact frame, and reading a run back calls no model.
- Small: a Python 3.13 library with no runtime dependencies and no hidden state.
Use it when you can name the premise and a consequence that should change without it: a changelog line, a contributing rule, a contract clause, an access grant. For a score or a ranking, use an ordinary evaluation.
What Gadfly says
Each input gets one finding, and a frame with several inputs gets one overall category over them. The categories are not scores; the first rule that matches names the category.
carried: ablation moved the observation as the promise allows, and every carriage check passed: presentation and the shams held, the bypass exposed its opening, and the frame says the observation is what the recipient receives.coupled but misaligned: the relation matters, but presentation or placement moved it, or ablation moved it where the promise disallows.cooled: ablation left the observation unchanged while a control showed the scorer can move, so this relation has no operative influence here.unresolved: there is not enough agreed evidence: a trial is missing or split, repeat is unstable, a sham moved too, the scorer cannot tell, or a check the category needs is still open.outside scope: the receiving owner declares the distinction unnecessary, and their own check of that exclusion is recorded.
Every finding keeps its evidence (why this category) apart from its obligations (what the trials did not settle), and an obligation is not a passed check.
Under the default carriage: all-checks, a category waits
for the checks it needs: carried for every carriage check, cooled for a
control that moved, outside scope for the owner's scope check. Until then
the finding is unresolved, and any other open check is listed beside the
category. A frame with carriage: listed names the category at once and
lists its open checks beside it.
Install
Gadfly is the midwife package on PyPI
(why),
for Python 3.13 or later. Its command, gadfly, goes on your path with
uv or pipx, and uvx runs it once without
installing it:
uv tool install midwife
gadfly --version
uvx --from midwife gadfly --help
To use it from Python, add it to your project with uv add midwife or
pip install midwife, then import midwife. The examples read records kept
in the repository, so to run them, clone it:
git clone https://github.com/mirzahusadzic/gadfly.git
cd gadfly
uv sync
Reading the kept examples back needs nothing more. A live run also needs a battery: a scorer on this machine that Gadfly asks each trial.
Set up a battery
Each shipped battery calls a service on this machine: install and start eGemma for Gemma 4, or the unified model router for Claude, DeepSeek and the other models it serves. Batteries has each one's address and key, the other shipped batteries, and your own.
Use
Every example is a real experiment: a frame, the predictions written before its first call, and the records of each run, read by Gemma 4 26B through eGemma, or for rungs 19 to 22 by Muse Spark 1.3 Contributor through the router, which in rung 20 also writes what it reads. Reading one back calls no model:
$ uv run python examples/read.py examples/07-sql-rule.json
sql-rule: 5 kept runs, read by google/gemma-4-26B-A4B-it-qat-q4_0-gguf@d1c082be
trial review
baseline request-changes 25
repeat request-changes 25
ablation request-changes 25
sham-1 request-changes 25
sham-2 request-changes 25
presentation request-changes 25
control approve 25
run 1: cooled
run 2: cooled
run 3: cooled
run 4: cooled
run 5: cooled
A pull request formats a value into SQL, which the repository's rules forbid.
Gemma asks for changes with the rule and without it, so its training decides
and the rule carries nothing here: cooled. Your first
frame climbs one frame, a trial at a time, to carried.
Run a frame
With a battery set up, these four commands check a frame, show its trials and their cost, run it, and read its verdict:
uv run gadfly check-frame examples/upgrade.json
uv run gadfly plan examples/upgrade.json
uv run gadfly run examples/upgrade.json \
--battery midwife.batteries.egemma:EGemma \
--calls 40 --out records.jsonl
uv run gadfly report examples/upgrade.json records.jsonl
check-frame and plan make no call, and run keeps each record as one JSON
line the moment it exists. Command line has every
command and flag, stats and recoil's walk, successor, firm and gate
among them.
From Python
The same test in Python reads a kept run back, then makes a fresh one:
import json
from pathlib import Path
from midwife.batteries.egemma import EGemma
from midwife.cli import load_records
from midwife.pipeline import calls_needed, judge, run, situate
from midwife.types import Budget
source = Path("examples/upgrade.json").read_text()
frame = situate(json.loads(source))
kept_run = load_records("examples/runs/upgrade/1.jsonl")
[finding] = judge(frame, kept_run)
print(finding.category, finding.evidence)
print(finding.obligations)
kept = []
budget = Budget(calls=calls_needed(frame))
run(frame, EGemma(), budget, on_record=kept.append)
[finding] = judge(frame, kept)
Learn more
The documentation site is built from docs/, and every page reads on GitHub
too.
- The examples: experiments that climb one capability a step, each with its predictions and kept runs.
- Write a frame: every key, and what the parser refuses.
- Reading a verdict: the rules in order, evidence apart from obligations, several inputs, and stopped runs.
- Statistics: how firm a verdict is over its kept runs, beside it and never changing it.
- Crossings: a decision made from something written from the evidence, with the maker inside the reading.
- Recoil: a piece found deep in a chain of frames, carried back across its crossings to the hole it fits, asked until firm.
- Batteries: the shipped scorers, their privacy guarantees, and writing your own.
- Architecture and limits: what Gadfly keeps, and what a verdict does not claim.
- Python API: every public function and class.
Cite
The working paper on Zenodo describes the method, this library and its examples' evidence. The paper is under CC BY 4.0 and the code under Apache-2.0; the paper's DOI always resolves to its latest version:
@misc{husadzic2026loadbearing,
author = {Husadzic, Mirza},
title = {When Meaning Is Load-Bearing: Testing Whether
a Decision Rests on Its Declared Evidence},
year = {2026},
publisher = {Zenodo},
note = {Working paper},
doi = {10.5281/zenodo.23962158},
url = {https://doi.org/10.5281/zenodo.23962158}
}
Develop
The suite needs no model and no network. These commands run it, lint, bring the docs' transcripts, copies and reference back in line with the code, and build the docs site with VitePress, on Node 24 or later:
uv sync
uv run pytest
uv run ruff check midwife examples tests tools
uv run ruff format --check midwife examples tests tools
uv run python tools/doc_outputs.py --write
npm ci
npm run docs:build
docs:build writes the site to docs/.vitepress/dist and fails on a dead
link, and npm run docs:dev serves one that follows your edits. The After the
Poem replay runs when GADFLY_ATP_PLANS names its plans.
Upgrading · Troubleshooting · Fonts and libraries · Apache-2.0 · The essay under CC BY 4.0
Metadata
Release files for midwife 0.12.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| midwife-0.12.0.tar.gz | 956.7 kB | With provenance | Details for midwife-0.12.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| midwife-0.12.0-py3-none-any.whl | Python 3 | none | any | Details for midwife-0.12.0-py3-none-any.whl |
Total release size: 1.1 MB
Release files / midwife-0.12.0.tar.gz
| Download URL | midwife-0.12.0.tar.gz |
|---|---|
| Size | 956.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
5a5d1fa722e58e7387a4f163a358091ba1cdca8203c1ace80df199a968de422f
|
|
BLAKE2b-256 checksum How to use checksums |
ce3a42cbd163d11f2ab110ac69074e53226406c7a7e37176ba80aac73a12111c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 5, 2026.
Transparency logRelease files / midwife-0.12.0-py3-none-any.whl
| Download URL | midwife-0.12.0-py3-none-any.whl |
|---|---|
| Size | 120.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
13ff5f3c15823270671c587c6e43918e0012f32e7d8da07f670406f39362e144
|
|
BLAKE2b-256 checksum How to use checksums |
b6610013a30df583beaa349b81a7f37c3ad9e8268d04f5cb2e7d2b587518b766
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 5, 2026.
Transparency logSecurity for midwife 0.12.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for midwife-0.12.0.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 5, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.