BIOMERO.shallower

🚀 This package is part of BIOMERO 2.0 — For complete deployment and FAIR infrastructure setup, start with the NL-BIOMERO Documentation 📖

A filesystem-only library and CPU container for shallow Zarr. It compares returned images and labels with the workflow's canonical inputs, replaces verified duplicate arrays with references, and retains new or changed data. No OMERO connection or credentials are required.

BIOMERO.importer uses the Python package locally. BIOMERO core runs the same implementation on Slurm before archiving and transferring results. Deduplication never suppresses result registration in OMERO.

The package writes the experimental BIOMERO shallow-manifest schema 2 for NGFF 0.4 / Zarr v2 Images and Plates. This private format is not RFC-8. Its portable image/label graph is separate from BIOMERO storage bindings so a future standards adapter can reuse it. Installing the package does not enable shallow storage; deployment configuration chooses local or remote shallowing.

Quick start

Python 3.11 or newer:

pip install 'biomero-shallower[identity]'
biomero-shallower --version
biomero-shallower inspect --returned-zarr /results/result.zarr

Container:

docker run --rm --network none cellularimagingcf/biomero-shallower:0.1.0 health

Reconstruct a full Zarr

materialize_shallow_zarr can restore a standalone Image or Plate Zarr on disk, without an OMERO script or connection. In an NL-BIOMERO deployment, open Python:

docker compose exec biomeroworker /opt/omero/server/venv3/bin/python

Replace the example paths below with paths inside the container:

from pathlib import Path
from biomero_shallower.result_zarr import (
    load_managed_storage_roots,
    resolve_shallow_registration,
    materialize_shallow_zarr,
)

mount = Path("/data")
source = Path("/data/Project B/.analyzed/WORKFLOW/TIMESTAMP/result.ome.zarr")
destination = Path("/data/Project B/reconstructed-result.ome.zarr")
roots = load_managed_storage_roots(
    import_mount_path=mount,
    config_file="/opt/omero/server/biomero-config.json",
)
view = resolve_shallow_registration(
    source, storage_roots=roots, import_mount_path=mount,
)
if view is None:
    raise ValueError("No shallow collection found at the source path")
result = materialize_shallow_zarr(view.reference, destination, roots)
print(result.destination)

Use the whole result root for a Plate. The destination must not exist, its parent must be writable, and there must be space for the full result. Original stores remain unchanged. All referenced pixels and labels must be accessible. The configuration path shown is the demo worker's group mapping configuration; custom deployments must supply their own authoritative mappings, including group_mappings_file if maintained separately. Outside the container, the same Python API works with matching packages and storage mappings.

Upgrade prerelease schema-1 stores

Schema-1 shallow stores created by prerelease builds are not loaded implicitly. Upgrade a settled store explicitly:

biomero-shallower migrate-v1 --returned-zarr /results/result.ome.zarr

For standalone stores, this converts the sidecar, report, and image metadata and keeps a sibling backup. Do not run it during an active transfer or import. For stores registered in OMERO, use the administrator script BIOMERO Migrate Shallow Storage so the filesystem and OMERO references are updated together.

See the reconstruction guide for details. A shallow result itself is not a self-contained OME-Zarr for generic readers; the reconstructed output is.

Documentation

• Usage and commands
• Architecture, recovery and Python API
• Development and testing
• Release setup

For local tests, install .[test] in a virtual environment and run python -m pytest tests -q. GitHub Actions runs unit tests, container smoke tests and a strict MkDocs build. GitHub releases publish the package and image.

License

Apache-2.0. See LICENSE. Dependencies retain their own licences.