scientific-data-contracts

sci-contract validates a computational biology dataset against a human-readable YAML contract and reports every failure it finds in one run — as a report for a person, or as JSON for a pipeline. It reads AnnData .h5ad files and SpatialData .zarr stores.

A contract states what a dataset must contain to be usable for a given analysis: which fields are present, what values they may take, and which scientific assumptions the data is expected to satisfy.

Why

A scientific dataset can be perfectly readable and still be unusable. The file opens, the array has the right shape, and the analysis runs — on cells whose condition label is spelled three different ways, against a segmentation from a different run, with no record of what produced any of it.

Those are scientific failures, not structural ones. Nothing in the file format can catch them, because nothing in the file format knows what the data is supposed to mean. A contract is where that gets written down: by the people who define the expectation, in a file the tooling that consumes the data can check.

Capabilities

A contract's dataset.type decides how the dataset is read. Nothing sniffs the path. Neither read is eager — the AnnData expression matrix stays on disk and SpatialData elements stay as unevaluated Dask graphs — with one exception, the opt-in instance check, which reads a segmentation raster because that is the only way to answer it.

Installation

Requires uv and Python 3.12 or newer.

The package is not yet on PyPI. Install it from a clone, which puts sci-contract on your PATH:

git clone https://github.com/ronfinn/scientific-data-contracts
uv tool install ./scientific-data-contracts
sci-contract --version

Or run it without installing anything, from a built wheel:

uv build
uvx --from dist/scientific_data_contracts-0.2.0-py3-none-any.whl sci-contract --version

To work on the tool rather than use it, sync the clone instead and prefix commands with uv run:

uv sync --dev
uv run sci-contract --version

Once the package is published, pip install scientific-data-contracts and uvx scientific-data-contracts will be the shorter routes. Neither works yet, and nothing above depends on them.

Quick start

Both example datasets are committed, so both commands work in a fresh clone.

uv run sci-contract validate examples/example.h5ad --contract examples/anndata-basic.yaml
uv run sci-contract validate examples/spatial-example.zarr --contract examples/spatialdata-basic.yaml

An abbreviated AnnData contract — obs and var columns, the values they may hold, and the provenance the file must carry:

version: 1

dataset:
  type: anndata

obs:
  required: [sample_id, condition, cell_barcode, pct_counts_mt]
  non_null: [sample_id, condition]
  unique:   [cell_barcode]
  allowed_values:
    condition: [control, treated]
  numeric_range:
    pct_counts_mt:
      min: 0
      max: 100

var:
  required: [gene_symbol]

provenance:
  required: [pipeline, pipeline_version, git_commit, container_digest]

And a SpatialData one — the elements, the space they share, and the table's link to the segmentation it annotates:

version: 1

dataset:
  type: spatialdata

spatial:
  required_elements:
    images: [morphology]
    labels: [cell_segmentation]
    tables: [table]

  required_coordinate_systems: [global]

  # Both elements must be registered into that system, not merely exist
  # alongside it.
  element_coordinate_systems:
    images:
      morphology: [global]
    labels:
      cell_segmentation: [global]

  # A table is not spatial by itself; it reaches coordinates through the region
  # it annotates.
  table_annotations:
    table:
      regions: [cell_segmentation]
      instances_must_exist: true

A rule may only name something the contract already requires — unique a required column, a table annotation a required table, an element registration a required element and a required coordinate system — because a rule about something the contract does not guarantee could never be evaluated.

Every field is specified in docs/contract-format.md.

Contract examples

Realistic contracts to read and adapt, in examples/contracts/:

• single-cell-anndata.yaml — a processed scRNA-seq dataset: experimental design, cell-type annotation, unique barcodes, quality measures within possible bounds, and provenance.
• xenium-like-spatialdata.yaml — an imaging-based spatial dataset: morphology image, cell and nucleus segmentation, transcript points, and a per-cell table resolved against the cell mask.
• visium-hd-like-spatialdata.yaml — a sequencing-based spatial dataset: tissue image, capture-bin geometry, and a per-bin table resolved against the bins.

The two spatial contracts are named after the assays that shape them, and that is where the resemblance stops. This tool validates a SpatialData store that already exists; it does not ingest a vendor bundle, and the element names in those files are the names a conversion step chose rather than standard ones. examples/contracts/README.md says what to change when adapting them.

Validation reports

A dataset that satisfies its contract, exit code 0:

$ uv run sci-contract validate examples/example.h5ad --contract examples/anndata-basic.yaml
scientific-data-contracts

Dataset   examples/example.h5ad
Contract  examples/anndata-basic.yaml

Dataset
  observations              6
  variables                 4

Contract
  required obs columns      3 / 3
  required var columns      1 / 1

Provenance
  required fields           4 / 4

Validation
  PASS

0 errors

One that does not, exit code 1. Every applicable check runs, so one invocation reports every failure rather than stopping at the first:

$ uv run sci-contract validate cohort.h5ad --contract cohort.yaml
...
Validation
  FAIL

Issues
  obs.cell_barcode          duplicate_obs_values
    contains duplicate values: AAACCTGAGAAACCAT
  obs.condition             invalid_obs_value
    contains values not allowed by the contract: placebo
  obs.pct_counts_mt         out_of_range_obs_value
    contains values outside the required range (0 to 100): -3, 130.25

3 errors

--format json renders the same result as a document with a stable schema_version and stable issue codes:

{
  "schema_version": 1,
  "valid": false,
  "dataset": "cohort.h5ad",
  "contract": "cohort.yaml",
  "dataset_type": "anndata",
  "observations": 6,
  "variables": 2,
  "required_obs_columns": { "required": 4, "present": 4 },
  "required_var_columns": { "required": 1, "present": 1 },
  "dataset_provenance": null,
  "spatial": null,
  "issues": [
    {
      "code": "duplicate_obs_values",
      "location": "obs.cell_barcode",
      "message": "contains duplicate values: AAACCTGAGAAACCAT"
    }
  ]
}

The two column-count objects are shown on one line each for brevity; the tool prints them expanded, and the issues array holds all three. Both renderings come from one result, so they cannot disagree. Every field and every code is specified in docs/report-format.md.

Pipeline integration

The JSON report is the interface. A workflow engine or CI job runs the validation, reads the report, and decides whether the dataset may be used.

flowchart TD
    D["dataset + contract"] --> S[sci-contract]
    S --> T[text report]
    S --> J[JSON report]
    J --> W["Nextflow / CI"]

examples/nextflow/ is a working Nextflow workflow that does exactly this: it validates a dataset, publishes the report either way, and fails the workflow when the dataset does not satisfy its contract, so the analysis step never runs on data that was rejected. It is two processes and a config file — no plugin, no container, no cloud profile.

nextflow run examples/nextflow
nextflow run examples/nextflow \
  --dataset  examples/spatial-example.zarr \
  --contract examples/spatialdata-basic.yaml \
  --report   spatial-report.json

With the package installed rather than run from a clone, add --sci_contract sci-contract.

Documentation

• docs/contract-format.md — every contract field, the errors an invalid contract produces, and the current limitations.
• docs/report-format.md — the report's JSON schema version, every field, every issue code, and the exit codes.
• docs/architecture.md — how the implementation is put together, how it is packaged, and what it deliberately avoids.
• examples/contracts/ — realistic contracts, and what they do and do not promise.
• examples/nextflow/ — the pipeline quality-gate example.
• CONTRIBUTING.md — the development loop.

Current release

v0.2.0. Distributable as a wheel and an sdist, with realistic single-cell and spatial-omics contract examples, uniqueness and numeric-range constraints, and a parameterised Nextflow gate. Contracts written for v0.1.0 remain valid and behave identically.

Roadmap

• Publish to PyPI, so installation is one command with no clone.
• Contracts for a SpatialData table's own obs and var.
• Richer AnnData constraints where a clear semantic model exists — the reasons dtypes are still absent are in docs/contract-format.md.
• Dataset checksums and stronger provenance.
• Deeper workflow integration driven by the JSON report.

What is deliberately not implemented is listed under "Current limitations" in docs/contract-format.md and docs/report-format.md.