Skip to main content

fairscape-conversion

Your metadata is already written down — as a datasheet, a datapackage, or a workflow engine's run output. Convert it into a FAIRSCAPE/EVI RO-Crate (ro-crate-metadata.json) instead of re-entering it. One command per format.

pip install -e .        # from this directory; installs fairscape-conversion

The fairscape import / fairscape export commands come with the fairscape CLI. Without it, every conversion also runs as python -m fairscape_conversion.core.cli convert <format> <import|export> IN [OUT].

Import — get an RO-Crate

Pick the row that matches what you have.

You have You need Run
A Datasheet for Datasets (D4D) the datasheet as YAML or JSON fairscape import d4d datasheet.yaml -o ./crate
A CFDE C2M2 datapackage the directory of TSVs + C2M2_datapackage.json fairscape import c2m2 ./datapackage-dir -o ./crate
A Workflow Run RO-Crate (CWL, Galaxy's RO-Crate export, …) its ro-crate-metadata.json fairscape import wrroc ro-crate-metadata.json -o ./crate
A Galaxy invocation the invocation export from Galaxy (Workflow Invocations → Export, any format: .tar.gz, .zip, .rocrate.zip, or unpacked), or just a .ga workflow file fairscape import galaxy invocation-export.tar.gz -o ./crate
A finished Cromwell/WDL run the file from cromwell run -m metadata.json fairscape import cromwell metadata.json -o ./crate
A finished Snakemake run the records JSON from snakemake --reporter fairscape fairscape import snakemake records.json -o ./crate
Finished MLflow runs the tracking store (an mlruns dir or tracking URI) and pip install mlflow fairscape import mlflow ./mlruns --experiment NAME -o ./crate
A REDCap project the data dictionary CSV (Project Setup → Data Dictionary → Download, or the API's content=metadata JSON), plus optionally the records export CSV fairscape import redcap MyStudy_DataDictionary.csv --records MyStudy_DATA.csv -o ./crate
A Frictionless Data Package (any datapackage.json) the package folder, or the descriptor fairscape import frictionless ./package-dir -o ./crate
A CPM RO-Crate (distributed provenance bundles) the crate directory: ro-crate-metadata.json + its CPMProvenanceFile PROV-N/PROV-JSON files fairscape import cpm ./crate-dir -o ./evi-crate

Export — from an RO-Crate

You want Run
A D4D datasheet fairscape export d4d ro-crate-metadata.json
A Workflow Run RO-Crate fairscape export wrroc ro-crate-metadata.json
An MLCommons Croissant document fairscape export croissant ro-crate-metadata.json
A CPM provenance document (PROV-JSON, or PROV-N with --provn) fairscape export cpm ro-crate-metadata.json
A Frictionless Data Package descriptor (datapackage.json; a Table Schema per tabular Dataset) fairscape export frictionless ro-crate-metadata.json

Try it — no data needed

Every conversion above has a runnable example in examples/, on real input that ships with the package. They need nothing installed:

python examples/run_all.py            # every one, with a pass/fail table
python examples/import_d4d.py         # or just the one you care about

Each prints what went in, what came out, the provenance edges it created, and whether the result still matches the plugin's reviewed golden file.

examples/mlflow/mlflow_to_rocrate.ipynb is the end-to-end walk-through: it trains a scikit-learn model with MLflow tracking, converts the store it just created, validates the crate, and renders its datasheet and provenance graph inline. A three-step pipeline with usedMLModel lives next to it in examples/mlflow/pipeline/. Both are checked in with their outputs.

The inputs and expected outputs the examples run on live inside each plugin:

format example input expected output
d4d plugins/d4d/input.yaml (the AI-READI datasheet) plugins/d4d/golden.json
c2m2 plugins/c2m2/input-datapackage/ (miniature datapackage) plugins/c2m2/golden.json
wrroc plugins/wrroc/input.json (CWL revsort run crate) plugins/wrroc/golden.json
cromwell plugins/cromwell/input.json (scatter workflow records) plugins/cromwell/golden.json
snakemake plugins/snakemake/input.json (3-rule chain records) plugins/snakemake/golden.json
mlflow plugins/mlflow/input.json (iris experiment records) plugins/mlflow/golden.json
croissant plugins/croissant/input.json (export this crate) plugins/croissant/golden.json
galaxy plugins/galaxy/input-store/ (Galaxy 23.0's export of the WRROC spec's collection workflow) plugins/galaxy/golden.json
redcap plugins/redcap/input-dictionary.csv + input-records.csv (a three-form public-health survey) plugins/redcap/golden.json
frictionless plugins/frictionless/input-datapackage/ (county surveillance: two tables with keys + a remote PDF) plugins/frictionless/golden.json
cpm plugins/cpm/input-crate/ (the CPM reference crate's provenance files, zenodo 7676924) plugins/cpm/golden.json (+ golden-export.json)

Linked crates — when one run's inputs were another run's outputs

Two independently converted crates do not know about each other. Pass the upstream crate to the conversion and the new crate reuses its identifiers for whatever it consumed from it, as a stub that points back:

fairscape import mlflow ./mlruns --experiment NAME -o ./crate --link-crate /path/to/upstream-crate
fairscape_conversion link ./crate --link-crate /path/to/upstream-crate    # or any crate, after the fact

The match is by file path, then md5, then containing directory; it runs after every importer, so it is not specific to any format. fairscape-artifacts follows the pointer and draws one evidence graph across both crates. The convention and two runnable pairs (Nextflow → MLflow, Snakemake → Snakemake) are in examples/linked-crates/.

From Python

import yaml
from fairscape_conversion.plugins import d4d

crate = d4d.convert("import", yaml.safe_load(open("datasheet.yaml")))

Same shape for every format: wrroc, c2m2, cromwell, snakemake, mlflow, cpm, redcap, frictionless, galaxy (convert("import", ...)), and d4d/wrroc/croissant/cpm/frictionless (convert("export", crate)). cpm takes the crate directory rather than a parsed document, because it reads the PROV bundle files registered in the metadata alongside it.

More

Metadata

Release files for fairscape-conversion 0.2.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for fairscape-conversion 0.2.1
File Size Uploaded
fairscape_conversion-0.2.1.tar.gz 251.2 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for fairscape-conversion 0.2.1
File Interpreter ABI Platform
fairscape_conversion-0.2.1-py3-none-any.whl Python 3 none any Details

Total release size: 532.8 kB

Release files / fairscape_conversion-0.2.1.tar.gz

Download URL fairscape_conversion-0.2.1.tar.gz
Size 251.2 kB
Tags Source
SHA-256 checksum
How to use checksums
814478ff256e5af44aa132fb3669911530a32b608051c1b901714e8fcc47d3fd
BLAKE2b-256 checksum
How to use checksums
e770d169bea3625827d56e9f9289ed1992e678ea99308b828cf0b7bad1e52019
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.14.6

Release files / fairscape_conversion-0.2.1-py3-none-any.whl

Download URL fairscape_conversion-0.2.1-py3-none-any.whl
Size 281.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
5d9d2233c96549d7cc339b7f73d4509782820839cb19998324992a974fc2c897
BLAKE2b-256 checksum
How to use checksums
83c2d231ab1e32d8b9fddb0cf1b23ca0fa12235300f932f09c0cc6ba7e08185b
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.14.6

Release history Release notifications | RSS feed

This release

0.2.1 This release

2 release files

0.2.0

2 release files

0.1.0

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page