OpenOmicsBench

OpenOmicsBench provides compact bulk RNA-seq count matrices for testing analysis software, teaching reproducible workflows and checking method behavior. Each biological object includes the source counts for its selected samples, a smaller pocket matrix, the sample design, source attribution, rights evidence, reference details and quantitative validation.

The version 1 collection contains 12 benchmark objects drawn from seven Expression Atlas studies. It covers human, mouse and Arabidopsis data, with balanced knockouts, paired tumour samples, factorial infection experiments, RNA interference and disease comparisons. The objects are tests of software and methods. They are not clinical reference data and do not replace the full source studies.

The published 0.1.0.dev0 prerelease records the earlier infrastructure baseline. Version 1.0.0 is the first complete collection release. The concept DOI always resolves to the latest archived version.

Quickstart

Use Python 3.11 or newer in an isolated environment:

python -m pip install openomicsbench
omicsbench list --assay bulk_rna_seq
omicsbench info rnaseq-002
omicsbench validate rnaseq-002

omicsbench validate checks the manifest, file inventory, byte counts, SHA-256 hashes, sample order, matrix shape and unchanged integer counts. It then recomputes the four declared preservation metrics. The installed package includes the complete collection, so these commands work without a repository checkout or network connection.

Use omicsbench get rnaseq-002 --size pocket to copy a verified tier into the local cache. Use omicsbench provenance rnaseq-002 to inspect its source and transformation record. Every public command includes an example in its help text.

Compare analysis results

Version 1.1 can compare results from another differential-expression tool with the full-source DESeq2 reference for a biological object:

omicsbench compare rnaseq-002 results.csv

The input may be comma-separated or tab-separated and must contain gene_id and log2_fold_change columns. Gene identifiers must be unique and effects must be finite. The JSON report gives rank correlation, top-50 overlap, sign agreement, coverage, missing and unexpected identifiers, and the threshold decision for each metric. A passing comparison exits with status 0. A completed comparison below the thresholds exits with status 2, allowing test pipelines to distinguish a scientific mismatch from invalid input.

The reference effects come from the same full-source DESeq2 1.50.2 runs used to certify the pocket matrices. The synthetic fixture has no DESeq2 reference and cannot be used with this command.

Version 1 collection

The catalog contains the complete object index. The tabular form is also available.

All 12 pockets pass the predeclared thresholds when compared with the corresponding full source matrix using DESeq2 1.50.2. The evidence files retain exact values, input hashes, workflow hashes, model designs and runtime versions. A separate reference check confirms that every pocket gene identifier occurs in the matching Ensembl or Ensembl Genomes annotation release.

Some objects share samples because they test different declared contrasts or strata from the same factorial study. The collection audit records those relationships and checks for conflicting IDs, cross-study sample collisions, unclassified duplicate files and repeated long prose.

Source data and licences

Expression Atlas and BioStudies supplied the biological source matrices. Each object includes attribution.json and rights.json. The distributed biological material is recorded as CC BY 4.0 with provider credit and a dated evidence link. Exact source, reference and transformation records sit beside the data rather than in a separate spreadsheet.

The software is licensed under Apache License 2.0. Project-written documentation and descriptive metadata are licensed under Creative Commons Attribution 4.0 International. Third-party material keeps the terms stated in its object record.

Project checks

Run the same checks used by continuous integration:

python -m unittest discover -s tests -v
python scripts/build_catalog.py
python scripts/audit_collection.py
python scripts/check_repository.py
python scripts/certify_release.py --preflight

The final v1 certification passes with no scientific, structural or publication blocker. A fresh Windows checkout passed the documented quickstart under Python 3.12.14. The owner approved that clean-room run as the publication check; no independent tester is claimed. The GitHub and Zenodo archives contain the same files, and every inventory-controlled file matches its certified byte count and SHA-256 value.

The version 1 release review brings the collection, scientific checks, overlap findings, software tests and remaining publication steps into one six-page document.

Citation

Zenodo should display the author as Vivaan Patni. GitHub development and commits use the account vxxqv. Cite the archived version used in an analysis; the concept DOI for all releases is 10.5281/zenodo.22551734.