daedalus-flow
Design, validate, and run labs of modules: reproducible analysis pipelines for data-intensive science.
daedalus-flow runs a lab.yaml recipe of Python modules in dependency order
and records provenance for every run. It has two main uses: compare methods on
one dataset, and run one analysis across many targets.
Status: alpha. The command surface and the on-disk lineage format may still change before 1.0.
Quickstart
dae example minimal # writes ./minimal/, a one-module lab
cd minimal && dae lab run # runs it under the built-in engine
dae flow status # reads the latest run back
The run writes a provenance tree under minimal/dae-outputs/, one folder per
step; the path is printed as lineage: when the run finishes. minimal needs
nothing beyond the base install, and dae with no arguments prints the next
step.
Install
pip install daedalus-flow
Requires Python 3.12 or newer. The base install pulls in no scientific stack. The extras add one dependency each:
| Extra | Adds | For |
|---|---|---|
demo |
numpy | the demo reference lab |
engine |
prefect | the Prefect backend, engine: prefect in lab.yaml |
viz |
grandalf | the graph layouts of dae lab visualize --style |
pip install "daedalus-flow[demo]"
Examples
dae example lists the bundled labs, simplest first; dae example <name>
writes ./<name>/ and prints the next command to run.
| Example | Teaches |
|---|---|
minimal |
one module, one step: the smallest lab that runs |
ensemble |
one input fanned across many targets |
parallel |
branches that run side by side, then a barrier join |
isolation-nix |
per-module nix environments, two pinned library versions |
complex |
a 30-module lab: nested and sibling collectors, a token-solo join |
demo |
the exo-survey reference lab: three targets, two fitting methods (needs the demo extra) |
dae module convert script.py scaffolds a module from an existing script.
Documentation
daedalus-flow.readthedocs.io holds the
tutorials, how-to guides and reference. The same pages live under
docs/, and dae --help covers the commands.
- Install and getting started
- Concepts: labs, modules, walks, flights and roles
lab.yamlreference, CLI and the--jsonenvelope- Contributing: setup and the
just gatecheck
How to cite
Cite the software with the entry below, or use the "Cite this repository"
button on GitHub, which reads CITATION.cff.
@software{daedalus-flow,
author = {Reller, Patricio},
title = {daedalus-flow: reproducible labs of modules for
data-intensive science},
year = {2026},
version = {0.1.0},
url = {https://github.com/preller/daedalus-flow}
}
Acknowledgments
daedalus-flow was developed by Patricio Reller
(ORCID 0000-0003-3161-9239) at the
Centre for Data Intensive Science and Industry, Department of Physics and
Astronomy, University College London. Bruno Merín and Ingo Waldmann advised on
the scientific direction. The work was co-funded by United Kingdom Research and
Innovation (UKRI) through University College London, and by the European Space
Agency (ESA).
License
Apache-2.0. The optional [viz] and [engine] extras pull in MPL-
and EPL-licensed packages.
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file daedalus_flow-0.1.0.tar.gz.
File metadata
- Download URL: daedalus_flow-0.1.0.tar.gz
- Upload date:
- Size: 713.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
b2ca19b25e37c06d4f4d5ce9e84435610d5d5d576065a6be745cc27f62e075b7
|
|
| MD5 |
a5dc5edf89370545c41bf0895d839275
|
|
| BLAKE2b-256 |
ab658ea5d888b890aecd672e29397f551844fc5d93b318d554b492b5a2595089
|
Provenance
The following attestation bundles were made for daedalus_flow-0.1.0.tar.gz:
Publisher:
publish.yml on preller/daedalus-flow
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
daedalus_flow-0.1.0.tar.gz -
Subject digest:
b2ca19b25e37c06d4f4d5ce9e84435610d5d5d576065a6be745cc27f62e075b7 - Sigstore transparency entry: 2680905437
- Sigstore integration time:
-
Permalink:
preller/daedalus-flow@a360156b776c8fa5408be47c563b56d7044e74f3 -
Branch / Tag:
refs/tags/v0.1.0 - Owner: https://github.com/preller
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish.yml@a360156b776c8fa5408be47c563b56d7044e74f3 -
Trigger Event:
push
-
Statement type:
File details
Details for the file daedalus_flow-0.1.0-py3-none-any.whl.
File metadata
- Download URL: daedalus_flow-0.1.0-py3-none-any.whl
- Upload date:
- Size: 266.6 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
8841abae32ce44ded9a2f637fa8aabeb353579d4258043ca97499aeb0d10fd72
|
|
| MD5 |
fdbf54baceb69bf33935c8c7e9675a60
|
|
| BLAKE2b-256 |
b99f5772bd17c2e637cf753b580ee2e68bd0a74d91f78e77934e0d0c157fe6aa
|
Provenance
The following attestation bundles were made for daedalus_flow-0.1.0-py3-none-any.whl:
Publisher:
publish.yml on preller/daedalus-flow
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
daedalus_flow-0.1.0-py3-none-any.whl -
Subject digest:
8841abae32ce44ded9a2f637fa8aabeb353579d4258043ca97499aeb0d10fd72 - Sigstore transparency entry: 2680905585
- Sigstore integration time:
-
Permalink:
preller/daedalus-flow@a360156b776c8fa5408be47c563b56d7044e74f3 -
Branch / Tag:
refs/tags/v0.1.0 - Owner: https://github.com/preller
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish.yml@a360156b776c8fa5408be47c563b56d7044e74f3 -
Trigger Event:
push
-
Statement type:
