fairscape-conversion
Your metadata is already written down — as a datasheet, a datapackage, or a
workflow engine's run output. Convert it into a FAIRSCAPE/EVI RO-Crate
(ro-crate-metadata.json) instead of re-entering it. One command per format.
pip install -e . # from this directory; installs fairscape-conversion
The fairscape import / fairscape export commands come with the
fairscape CLI. Without it, every conversion also runs as
python -m fairscape_conversion.core.cli convert <format> <import|export> IN [OUT].
Import — get an RO-Crate
Pick the row that matches what you have.
| You have | You need | Run |
|---|---|---|
| A Datasheet for Datasets (D4D) | the datasheet as YAML or JSON | fairscape import d4d datasheet.yaml -o ./crate |
| A CFDE C2M2 datapackage | the directory of TSVs + C2M2_datapackage.json |
fairscape import c2m2 ./datapackage-dir -o ./crate |
| A Workflow Run RO-Crate (CWL, Galaxy's RO-Crate export, …) | its ro-crate-metadata.json |
fairscape import wrroc ro-crate-metadata.json -o ./crate |
| A Galaxy invocation | the invocation export from Galaxy (Workflow Invocations → Export, any format: .tar.gz, .zip, .rocrate.zip, or unpacked), or just a .ga workflow file |
fairscape import galaxy invocation-export.tar.gz -o ./crate |
| A finished Cromwell/WDL run | the file from cromwell run -m metadata.json |
fairscape import cromwell metadata.json -o ./crate |
| A finished Snakemake run | the records JSON from snakemake --reporter fairscape |
fairscape import snakemake records.json -o ./crate |
| Finished MLflow runs | the tracking store (an mlruns dir or tracking URI) and pip install mlflow |
fairscape import mlflow ./mlruns --experiment NAME -o ./crate |
| A REDCap project | the data dictionary CSV (Project Setup → Data Dictionary → Download, or the API's content=metadata JSON), plus optionally the records export CSV |
fairscape import redcap MyStudy_DataDictionary.csv --records MyStudy_DATA.csv -o ./crate |
A Frictionless Data Package (any datapackage.json) |
the package folder, or the descriptor | fairscape import frictionless ./package-dir -o ./crate |
| A CPM RO-Crate (distributed provenance bundles) | the crate directory: ro-crate-metadata.json + its CPMProvenanceFile PROV-N/PROV-JSON files |
fairscape import cpm ./crate-dir -o ./evi-crate |
Export — from an RO-Crate
| You want | Run |
|---|---|
| A D4D datasheet | fairscape export d4d ro-crate-metadata.json |
| A Workflow Run RO-Crate | fairscape export wrroc ro-crate-metadata.json |
| An MLCommons Croissant document | fairscape export croissant ro-crate-metadata.json |
A CPM provenance document (PROV-JSON, or PROV-N with --provn) |
fairscape export cpm ro-crate-metadata.json |
A Frictionless Data Package descriptor (datapackage.json; a Table Schema per tabular Dataset) |
fairscape export frictionless ro-crate-metadata.json |
Try it — no data needed
Every conversion above has a runnable example in examples/,
on real input that ships with the package. They need nothing installed:
python examples/run_all.py # every one, with a pass/fail table
python examples/import_d4d.py # or just the one you care about
Each prints what went in, what came out, the provenance edges it created, and whether the result still matches the plugin's reviewed golden file.
examples/mlflow/mlflow_to_rocrate.ipynb
is the end-to-end walk-through: it trains a scikit-learn model with MLflow
tracking, converts the store it just created, validates the crate, and renders
its datasheet and provenance graph inline. A three-step pipeline with
usedMLModel lives next to it in examples/mlflow/pipeline/.
Both are checked in with their outputs.
The inputs and expected outputs the examples run on live inside each plugin:
| format | example input | expected output |
|---|---|---|
| d4d | plugins/d4d/input.yaml (the AI-READI datasheet) |
plugins/d4d/golden.json |
| c2m2 | plugins/c2m2/input-datapackage/ (miniature datapackage) |
plugins/c2m2/golden.json |
| wrroc | plugins/wrroc/input.json (CWL revsort run crate) |
plugins/wrroc/golden.json |
| cromwell | plugins/cromwell/input.json (scatter workflow records) |
plugins/cromwell/golden.json |
| snakemake | plugins/snakemake/input.json (3-rule chain records) |
plugins/snakemake/golden.json |
| mlflow | plugins/mlflow/input.json (iris experiment records) |
plugins/mlflow/golden.json |
| croissant | plugins/croissant/input.json (export this crate) |
plugins/croissant/golden.json |
| galaxy | plugins/galaxy/input-store/ (Galaxy 23.0's export of the WRROC spec's collection workflow) |
plugins/galaxy/golden.json |
| redcap | plugins/redcap/input-dictionary.csv + input-records.csv (a three-form public-health survey) |
plugins/redcap/golden.json |
| frictionless | plugins/frictionless/input-datapackage/ (county surveillance: two tables with keys + a remote PDF) |
plugins/frictionless/golden.json |
| cpm | plugins/cpm/input-crate/ (the CPM reference crate's provenance files, zenodo 7676924) |
plugins/cpm/golden.json (+ golden-export.json) |
Linked crates — when one run's inputs were another run's outputs
Two independently converted crates do not know about each other. Pass the upstream crate to the conversion and the new crate reuses its identifiers for whatever it consumed from it, as a stub that points back:
fairscape import mlflow ./mlruns --experiment NAME -o ./crate --link-crate /path/to/upstream-crate
fairscape_conversion link ./crate --link-crate /path/to/upstream-crate # or any crate, after the fact
The match is by file path, then md5, then containing directory; it runs after
every importer, so it is not specific to any format. fairscape-artifacts
follows the pointer and draws one evidence graph across both crates. The
convention and two runnable pairs (Nextflow → MLflow, Snakemake → Snakemake)
are in examples/linked-crates/.
From Python
import yaml
from fairscape_conversion.plugins import d4d
crate = d4d.convert("import", yaml.safe_load(open("datasheet.yaml")))
Same shape for every format: wrroc, c2m2, cromwell, snakemake,
mlflow, cpm, redcap, frictionless, galaxy (convert("import", ...)), and d4d/wrroc/croissant/cpm/frictionless
(convert("export", crate)). cpm takes the crate directory rather than a
parsed document, because it reads the PROV bundle files registered in the
metadata alongside it.
More
- Your format isn't listed →
docs/NEW-PLUGIN.md— a converter is a folder of two CSVs plus a small plugin class. - How the engine works, what's tested →
docs/INTERNALS.mdandMAPPING-SCHEMA.md.
Metadata
Release files for fairscape-conversion 0.2.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| fairscape_conversion-0.2.0.tar.gz | 250.3 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| fairscape_conversion-0.2.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 531.4 kB
Release files / fairscape_conversion-0.2.0.tar.gz
| Download URL | fairscape_conversion-0.2.0.tar.gz |
|---|---|
| Size | 250.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
608788fe288af4715987b605c2012663f66c0a4d848fb191a02f329f17dc2ca1
|
|
BLAKE2b-256 checksum How to use checksums |
4a6caae0d63f90e54fd4836918d342743c6ee5fa362ac41e1827ee507cf3303b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.14.6
|
Release files / fairscape_conversion-0.2.0-py3-none-any.whl
| Download URL | fairscape_conversion-0.2.0-py3-none-any.whl |
|---|---|
| Size | 281.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
1643aedd102da67a22a241d7dae15812c7f99063aae0697e874f9a06c8c4c8cd
|
|
BLAKE2b-256 checksum How to use checksums |
f37afe31d8fd5e3e9a4dce0541ca6d4845b9e8d47f973573fccfe68d173e0c27
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.14.6
|