snakemake-report-plugin-fairscape

A Snakemake report plugin that turns a completed run into a FAIRSCAPE EVI RO-Crate — ro-crate-metadata.json describing every job, rule, and file of the run, plus the derived artifact set (datasheet, AI-Ready score, evidence graph, LinkML/D4D export, inferred schemas). No changes to your workflow: run Snakemake as usual, then ask for the report.

pip install 'snakemake-report-plugin-fairscape[artifacts]'

cd your-workflow/
snakemake --cores 4             # run as usual
snakemake --reporter fairscape  # writes ro-crate-metadata.json + artifacts

The [artifacts] extra pulls in fairscape-cli, which generates everything downstream of the crate. Those steps — datasheet, evidence graph, LinkML export and schema inference — need fairscape-cli 1.2.10 or newer, the release that exposes process_crate and infer_schema as importable functions; the extra pins it. Installed without the extra, the crate itself is still written and every step after it is skipped with a note.

Where the code lives

This repository is the Snakemake side only. It reduces Snakemake's report interface, persistence metadata, and filesystem state to a plain-data records document, and hands that to the shared converter engine:

Nothing downstream of the crate is reimplemented here — the reporter calls fairscape_cli.utils.build_utils.process_crate and fairscape_cli.models.schema.infer_schema directly, so parity with the CLI is by construction rather than by test suite. The mapping in fairscape_conversion is CSV-driven and pinned by a golden-file test built from this repo's letters-chain example.

Set SNAKEMAKE_FAIRSCAPE_DUMP_RECORDS=<path> on a report run to capture the records document that crosses that boundary (debugging, fixture refresh).

Options

All optional; every flag is prefixed --report-fairscape-.

The on-by-default artifacts read only the crate JSON; everything that touches data files is opt-in.

Derived artifacts

After writing the crate, the reporter runs fairscape-cli's single-crate build pipeline in the CLI's own order (link inverses → EVI:inputs/EVI:outputs → evidence graph → LinkML → datasheet).

Schema ARKs are minted deterministically (hashed from the Dataset's ARK, overriding the CLI's random uuid via infer_schema(guid=...)), so the ARK-stability guarantee below survives --report-fairscape-schemas.

What ends up in the crate

Mirrors nf-fairscape's emission:

Both edge directions are written (generated and generatedBy).

ARKs are ark:{naan}/{prefix}-{slug}-{sha1(source)[:7]} and hash only run-independent strings (Snakefile path, rule name, rule + sorted outputs, file path) — so re-running the report, and even fully re-executing the workflow (--forceall), reproduces identical identifiers. tools/run-examples.sh asserts this.

Files under the crate directory get a crate-relative contentUrl; files outside it — or already deleted, like temp() intermediates — get localPath ("was here when this ran").

What a rule's Software contentUrl points at

script:/notebook:/wrapper: jobs record no shell command in the persistence metadata, so their Computation's command falls back to script: scripts/x.py (etc.) instead of being empty.

A script invoked through shell: "python scripts/foo.py" is not found — Snakemake treats the path as an opaque token in the shell string. It appears in the crate only if also listed under input:, and then as a Dataset, not Software.

Which files the crate sees

Every declared output of every executed job — nothing needs to be flagged. The report interface carries two separate channels: results holds only files marked report(...) in the workflow (that's what the HTML reporter showcases), while jobs covers the whole DAG — every job record lists every file in its rule's output:, and its input: list comes from the persistence metadata. This plugin reads jobs and ignores results, so intermediates are captured automatically; none of the example workflows use report() at all.

Final outputs are derived, not declared: the run Computation's generated is the set of outputs no other job consumed. That reproduces "what the user asked for" in the normal case (the target rule all has no outputs, so it produces no job record — its inputs are exactly the terminal outputs). Caveat: a requested file that is also consumed by another job counts as an intermediate here and drops out of the run's generated (its Dataset and edges remain).

Why post-hoc?

Snakemake offers three places to hook provenance capture; this plugin uses the first, which is also what the official WRROC plugin (snakemake/snakemake-report-plugin-rocrate, early-stage, unreleased) chose.

1. Report plugin, separate invocation (this plugin). Snakemake persists per-job metadata (resolved shell command, inputs, params, conda env, container, times) into .snakemake/metadata during every run; --reporter fairscape re-reads it afterwards. Pros: zero runtime risk and zero overhead (a crash in the reporter can never fail or slow a run); works on runs that finished before the plugin was installed; re-runnable with different options; the sanctioned extension point built exactly for this. Cons: a second command to remember; sees only what persistence kept — no original CLI line, no failed jobs (a failed job writes no metadata beyond an incomplete flag), temp() files may already be deleted (their Datasets lose contentSize), and a later run of the same workflow overwrites the metadata (last run wins). The Snakefile must still parse at report time.

2. Report plugin, same invocation (--report-after-run). Snakemake can run the reporter automatically at the end of the run — but as of snakemake 9.25.1 the CLI wiring blocks custom reporters: --report-after-run requires --report, and --report force-selects the html reporter (snakemake/cli.py:2071-2078). If that upstream wiring is relaxed, this plugin would work there unchanged — same data, just triggered automatically.

3. Logger plugin (live, during execution). The only genuinely during-execution extension point (snakemake-interface-logger-plugins) receives log events as jobs start and finish — the only place that could see failed jobs or the true command line. Cons: it gets log records, not the DAG/persistence data contract, so the whole crate assembly would have to be reconstructed from event payloads; it runs in-process where an error can disturb the run; and it's a logging API, not a provenance one. Nextflow needs the observer approach because it keeps no queryable per-task metadata after the run — Snakemake does, which is exactly what makes post-hoc viable and simpler here.

Undeclared interfaces this relies on

The declared report-plugin contract (ReporterBase, snakemake-interface-report-plugins 1.3.0) exposes rules/jobs/configfiles/dag, but three load-bearing pieces are outside it (each wrapped so absence degrades the crate rather than failing the report):

• dag.workflow.persistence.metadata(f) → MetadataRecord (record format version 6, snakemake/persistence/__init__.py) — the only source of the resolved shellcmd, input list, and params per job.
• dag.workflow.main_snakefile — the workflow identity that ARKs hash.
• job_record.job.wildcards_dict — only .rule is declared on the live job.

Developed and tested against snakemake 9.25.1 / snakemake-interface-report-plugins 1.3.0.

Known limits

• The run Computation's command is literally snakemake — the original command line is not recorded in the metadata the report step can see (see "Why post-hoc?").
• Jobs that failed or were never run leave no persistence metadata and are absent from the crate (only successful work is described).
• A rule with no outputs (like a bare all target) produces no job record and no Computation.
• No per-file checksums yet; md5 fields are absent from Datasets, which costs the AI-Ready verifiable criterion.

Development

pip install '.[artifacts,validate]'    # NOT -e: see below
pip install snakemake
tools/run-examples.sh

Install non-editable. Snakemake's plugin registry discovers plugins with pkgutil.iter_modules, which does not see setuptools editable installs — a pip install -e . plugin is silently absent from --reporter. Re-run pip install . after each change.

tools/run-examples.sh asserts the derived artifacts only when the installed fairscape-cli can produce them, so it stays green against an older one while saying what it skipped. It runs all four examples (letters-chain — 3-rule linear chain; sample-fanout — wildcard fan-out over 3 samples + aggregation + configfile + root inputs + --report-fairscape-schemas; python-script — a script: rule, proving the Software contentUrl points at the .py file; dir-plots — a directory() output with --report-fairscape-expand-directories), asserts every derived artifact exists, validates each crate with fairscape_models.ROCrateV1_2.model_validate plus a dangling-ARK check (tools/validate_crate.py), and asserts ARK stability across a second report invocation. CI runs exactly this.

Releasing a new version: see RELEASING.md.

License

Apache-2.0. See LICENSE.