appfl-bio-suite
Cross-silo federated learning experiments in computational biology, built on APPFL, hivewatch and Globus Compute.
Institutions train on their own data, on their own hardware, and return only results. The standard multi-user deployment uses individual identities, with no shared credential, VPN, or inbound network access.
Experimental: single-user service-account onboarding. An opt-in prototype uses one service identity per site, with credentials held by that site and the coordinator. It is not yet the default or a production-ready replacement for multi-user endpoints. See the trial guide and design evaluation.
→ mfjoel01.github.io/appfl-bio-suite — the project site, with a live preview of the federation network map.
YOU (coordinator) PARTNER SITES
┌──────────────┐
│ driver │ ──── task ────► their cluster, their data
│ aggregator │ ◄─── result ─── (weights, or summary statistics)
└──────────────┘
The experiments
Fine-mapping also has a completed optional
MultiSuSiE coordinator pipeline and
matched-filter comparison against SuSiEx.
Use run fine-mapping --backend susiex or --backend multisusie with either the
local or remote driver; --results-dir keeps their outputs separate.
The comparison report and figures
cover both methods across cohorts and filter policies.
Four experiments, one toolchain. The first three dispatch to partner sites; the fourth is in development.
| What it is | Data | |
|---|---|---|
| FLamby Fed-Heart-Disease | Multi-round FedAvg on a public tabular benchmark with natural hospital splits | Partners download it; ~40 KB, CC BY 4.0 |
| Federated GWAS | Single-round summary-statistic meta-analysis with polygenic-score evaluation | Simulated and distributed by the coordinator |
| Federated fine-mapping | Single-round cross-ancestry credible-set construction from per-site second moments — provably equal to a pooled fit, not an approximation of one. Governed by GA4GH DUO, DRS, TRS and TES | Simulated and distributed by the coordinator |
| Federated methylation · work in progress | Multi-round FedAvg for methylation-based subtype classification on sparse, nanopore-like input, against pooled and local-only training. Currently a single-machine simulation | Synthetic, or open TCGA-LAML methylation data from the NCI GDC |
Federated methylation is a work in progress. All of its sites are simulated in one process today, so there is no partner bundle or endpoint for it yet. Its results so far, including one real-data run, are recorded as measured in its record of runs.
Quickstart
Python 3.12 is required. Install the release from PyPI, with the release's pinned constraints:
python3.12 -m venv .venv && source .venv/bin/activate
python -m pip install "appfl-bio-suite[all]==0.3.2" \
-c https://raw.githubusercontent.com/mfjoel01/appfl-bio-suite/v0.3.2/constraints.txt
Always pin the exact version. Every site in a federation must run the same release, and
-c pins the rest of the dependency tree to the versions that release was tested with.
Prove the install works — a complete federation on one machine, no partners, no network, and nothing to fill in first:
appfl-bio-suite simulate gwas --scenario ci-tiny --out /tmp/gwas-ci
appfl-bio-suite run gwas --config loopback --driver serial --data-root /tmp/gwas-ci
Results land in local/output/gwas-loopback/ under the current directory. This is the
same pair of commands CI runs.
Then stand up a real federation. The example federation file, the fine-mapping binary
installers under scripts/, and the example configurations are not part of the wheel,
so take them from the matching release tag:
git clone --branch v0.3.2 https://github.com/mfjoel01/appfl-bio-suite.git
cd appfl-bio-suite
cp federation.yaml.example local/federation.yaml
$EDITOR local/federation.yaml
appfl-bio-suite preflight
To modify the suite, install from that checkout instead:
pip install -e ".[all]" -c constraints.txt. Fine-mapping with the SuSiEx backend also
needs two vendored binaries on PATH; the guide below covers both install routes.
→ docs/coordinator/new-federation.md takes you from here to a running experiment across institutions you recruit yourself.
The CLI
appfl-bio-suite preflight [--experiment X] environment, pins, configs, endpoints
appfl-bio-suite endpoint status [SITE] is it online
appfl-bio-suite endpoint smoke [SITE] round-trip: does it actually work
appfl-bio-suite identity validate <mapping.json> catch mapping bugs offline, in a second
appfl-bio-suite simulate EXPERIMENT --scenario NAME generate and split synthetic data
appfl-bio-suite run EXPERIMENT run it
appfl-bio-suite partner-bundle EXPERIMENT --site X a partner's complete setup package
appfl-bio-suite watch build | serve | export the federation, on a map
appfl-bio-suite ga4gh duo | drs | trs | tes data use, data objects, tools, tasks
How it works
One config file. federation.yaml declares who you are, who your partners are, and
what each is doing. Every generator, config, document template and check reads from it.
Nothing in src/ contains a coordinator identity, an endpoint UUID, or a site name, and
CI verifies that.
Generated partner bundles, not templates. partner-bundle emits a directory with a
partner's configuration fully resolved — their site ID, their data assignment, your
identity, their scheduler's provider block — plus their two setup documents with no
placeholders left to interpret.
A closed set for partner-facing content. Bundles are built from docs/partner/ and
nowhere else, enforced in code, so nothing outside that tree can reach a partner.
Exact version pinning. Every site runs byte-identical versions of the Globus Compute
stack. Skew does not fail at install time; it fails as a deserialization error several
rounds into a run on somebody else's cluster. constraints.txt pins it and
preflight --check pins verifies it.
Failures that name the fix. The troubleshooting registry is indexed by symptom and gives the mechanism alongside the fix.
GA4GH standards
The fine-mapping experiment is assembled out of four GA4GH specifications, each closing a hole the design otherwise had.
| answers | ||
|---|---|---|
| DUO | Data Use Ontology | May this site's data be used for this study? |
| DRS | Data Repository Service | Exactly which bytes did a site compute over? |
| TRS | Tool Registry Service | Exactly which tool version computed them? |
| TES | Task Execution Service | Run that tool, there, on those bytes. |
Together: a named tool version, pinned by checksum, runs over content-addressed data whose use terms permit the study that dispatched it. Every clause of that sentence used to be an assumption.
A site enforces its own consent code. Each bundle carries a DATA_USE.json, and the
site's worker matches it against the study the coordinator declared — refusing before it
opens a genotype file, on its own hardware, in the account it controls. The coordinator
runs the same check offline before dispatch, but that one only saves a queue wait.
Bundles are content-addressed. A sha-256 per file and a Merkle id over the set, so "did this site run on the bundle I cut for it" is a string comparison. Without it, a site running last month's bundle produces well-formed aggregates over the wrong individuals, pools without complaint, and is wrong in no visible way.
Results name the code that made them. The tool pin covers the source of the two
modules APPFL ships to workers, is verified at launch, and is written into every run's
ga4gh_provenance.json alongside what each site attests to having actually done.
appfl-bio-suite ga4gh duo check --experiment fine-mapping # will every site permit this?
appfl-bio-suite ga4gh drs register --data-root local/data/… # address the bundles
appfl-bio-suite ga4gh trs publish --out local/trs # pin the tool
appfl-bio-suite ga4gh tes task --site X # what would run, as a document
All of it is opt-in per experiment: a federation.yaml with no ga4gh blocks produces
byte-identical configs to the ones it produced before any of this existed.
The network map
Everyone in the federation — coordinator and partners alike — can open the same page: every institution on a world map, which experiments each runs, what kind of data each holds, how many samples they declared, and whether their endpoint is answering.
pip install "appfl-bio-suite[watch]==0.3.2" -c constraints.txt # included in [all]
appfl-bio-suite watch build # draw it from federation.yaml
appfl-bio-suite watch serve # look at it locally
appfl-bio-suite watch export --out site/ # three static files anyone can host
appfl-bio-suite run gwas --watch # a live run, on the same map
Built on hivewatch, APPFL's monitoring toolkit, which supplies the event schema, the map format and the viewer.
The map exists before any run does. It is drawn from federation.yaml alone, so the
day the config names a partner, that partner is on the map — which is what makes it a
picture of the network rather than a dashboard for one job.
Site coordinates are declared in federation.yaml, never resolved from an IP address: a
partner's compute node has no outbound web access, and you already know where your
partners are. A site with no coordinates is reported as unplaced rather than drawn at a
guessed position. Service accounts, cluster paths, your Globus identity and (by default)
full endpoint UUIDs never reach the published page.
→ docs/coordinator/network-map.md
Reproducibility
- Every simulation run writes a manifest recording the scenario, every seed, the suite
commit, input and output checksums, and package versions.
simulate gwas --verify <manifest>confirms a rerun mechanically. - The GWAS simulation is deterministic. For a fixed seed, all 35 outputs are byte-identical, verified against recorded checksums.
- Known limits are documented. docs/experiments/gwas/DATA.md states which input cohort is not regenerable, and what the shipped substitute does and does not reproduce.
Documentation
Organized by audience.
| docs/README.md | Index: which document, for whom, in what order |
| docs/coordinator/ | Running a federation |
| docs/partner/ | What a partner receives — a closed set |
| docs/experiments/ | Per-experiment design, data, and runbooks |
Requirements
Python 3.12. A Globus account. Partners willing to run a privileged process on their own clusters.
Not required: a shared filesystem, a VPN, matching operating systems or schedulers, or inbound network access.
Licence
MIT. See LICENSE — and note the third-party terms recorded there for APPFL, FLamby, and PGS Catalog data.
For automated partner setup, see bundle-based onboarding.
Metadata
Release files for appfl-bio-suite 0.3.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| appfl_bio_suite-0.3.2.tar.gz | 1.3 MB | With provenance | Details for appfl_bio_suite-0.3.2.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| appfl_bio_suite-0.3.2-py3-none-any.whl | Python 3 | none | any | Details for appfl_bio_suite-0.3.2-py3-none-any.whl |
Total release size: 2.5 MB
Release files / appfl_bio_suite-0.3.2.tar.gz
| Download URL | appfl_bio_suite-0.3.2.tar.gz |
|---|---|
| Size | 1.3 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
50f5277e18d6ffe242100d1aee804f62962319ef8739700469c4696ec7d9c7f5
|
|
BLAKE2b-256 checksum How to use checksums |
2a1d5e1c4b381cec4e680785a10099ac99ebd8e6ef17a80e30fb2493dcb65401
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 11, 2026.
Transparency logRelease files / appfl_bio_suite-0.3.2-py3-none-any.whl
| Download URL | appfl_bio_suite-0.3.2-py3-none-any.whl |
|---|---|
| Size | 1.2 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
69671784dbd9e785097aa1671b8603fed8cd67fe298831d8ca8649a4a0c10773
|
|
BLAKE2b-256 checksum How to use checksums |
e2849dd0563c0db1eca19ddb5a3902eebc37d060a17cf8912e9fe36db582d39d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 11, 2026.
Transparency logSecurity for appfl-bio-suite 0.3.2
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for appfl_bio_suite-0.3.2.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 11, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.