aind-zarr-utils
Utilities for working with AIND Zarr assets, metadata, Neuroglancer annotations, and SmartSPIM pipeline coordinate transforms.
Recommended API
The primary entry point is Asset. It discovers an asset's alignment-channel
Zarr, loads acquisition metadata and processing.json, opens the Zarr once, and
reuses that state for image, stub, and point-transform workflows.
from aind_zarr_utils import Asset, Points, Space
from aind_s3_cache.json_utils import get_json
zarr_uri = "s3://aind-open-data/dataset/image.ome.zarr/0"
ng_state = get_json("s3://aind-open-data/dataset/neuroglancer_state.json")
asset = Asset.from_zarr(zarr_uri)
points = Points.from_neuroglancer(ng_state)
ccf = asset.transform(points, to=Space.CCF_MM)
print(ccf.values)
If you already have metadata and processing dictionaries loaded, use the no-I/O
constructor. alignment_zarr_uri should be the Zarr used by the alignment
pipeline; source_zarr_uri is optional provenance for the Zarr you started
from.
from aind_zarr_utils import Asset
asset = Asset(
alignment_zarr_uri="s3://bucket/asset/alignment.ome.zarr/0",
metadata=metadata,
processing=processing,
source_zarr_uri="s3://bucket/asset/acquisition.ome.zarr/0",
)
Metadata Reader Integration
This branch uses aind-metadata-reader to interpret historical and newer
acquisition and processing layouts. Asset discovery reads metadata.nd.json
when present and falls back to acquisition.json when the aggregate is absent.
Malformed JSON and access failures propagate. This file preference preserves
existing behavior; callers who need a different source can load it explicitly.
Anatomical conversion matches recorded coordinate-axis names to Zarr axis names.
It requires one cardinal frame; multiple frames require explicit selection.
Asset and all three discovery constructors accept an acquisition reader view
and a registration process:
from aind_metadata_reader import Document, read_acquisition, read_processing
from aind_zarr_utils import Asset
acquisition = read_acquisition(Document(metadata), device_name="SmartSPIM", frame_scope="local")
registrations = read_processing(Document(processing)).find("Image importing")
for run in registrations:
print(run.document.pointer, run.pipeline_name, run.pipeline_version, run.code_version)
# Choose a source pointer after inspecting the runs; a channel can have multiple runs.
registration = next(run for run in registrations if run.document.pointer == registration_pointer)
asset = Asset(
alignment_zarr_uri=zarr_uri,
metadata=metadata,
processing=processing,
acquisition=acquisition,
registration=registration,
)
image = asset.image(level=0)
pipeline_stub, size = asset.stub(pipeline=True)
Choose the device and frame that describe the Zarr. This selection does not
compose tile transforms or establish a global origin. To select a frame by its
exact source pointer, pass the CoordinateFrame returned by
read_acquisition(document).one_frame(pointer=frame_pointer) as acquisition.
The low-level image functions also accept either kind of view. The legacy
direction_from_* helpers still require recorded array dimensions, which newer
coordinate frames may omit.
The selected acquisition supplies geometry; the original asset.metadata
mapping remains the input to overlay predicates. The selected registration
supplies the component version and pipeline association used for overlays and
transform paths. Both views retain their source pointers, and asset.processing
keeps the original processing mapping. An explicit selection takes precedence
over automatic selection without rewriting the source metadata.
Registration component versions continue to select pipeline overlays independently of the metadata schema version. The reader's SmartSPIM helper interprets the observed registration 0.0.37 input convention; unverified producer conventions remain unresolved. Repeated registration steps require an unambiguous selection. Reading metadata does not certify schema validity. Pyramid translation corrections are explicit, as described below.
The reader is installed from PyPI with aind-metadata-reader>=0.2.3,<0.3.
Run uv sync to install the locked dependencies; a sibling checkout and internal
GitHub access are unnecessary. The upper bound keeps this integration within the
reader's tested experimental release series.
Physical Geometry And Pyramid Translations
Images and stubs use the recorded NGFF scales and translations, including
image-level transforms. Missing translations mean zero. Origin.default()
honors that recorded geometry; Origin.at(...) and Origin.at_corner(...)
rebase the full-resolution grid while preserving relative level translations.
Some SmartSPIM fusion outputs have missing pyramid translations or store voxel counts where physical units are declared. Apply the reader's correction rule when you have evidence that the selected fusion process produced this Zarr and that its pyramid uses aligned block averaging:
asset = Asset.from_root(asset_uri)
corrected = asset.with_pyramid_translation_correction(
author="your-name",
association_evidence=(association_check_reference,),
averaging_evidence=(block_averaging_check_reference,),
)
image = corrected.image(level=3)
decision = corrected.geometry.evaluations[0]
manifest = corrected.geometry.manifest()
The evidence references should identify the checks performed for this artifact.
The fusion process is selected by its recorded output channel when unique;
pass producer=selected_process to resolve multiple runs. The rule diagnoses the
observed defect signature even for unfamiliar producer versions. It assumes no
first-fixed version: correct translations produce not_applicable, and other
patterns remain unknown with a warning and the original geometry.
with_pyramid_translation_correction returns a new asset carrying the reader's
interpretation and linked rule evaluation. It changes the physical geometry used for images and stubs
without changing stored attributes, pixels, acquisition metadata, or processing
records. zarr_attributes exposes the original parsed attributes; geometry
retains the original/derived snapshots and replayable manifest. The interpretation
is bound to the source locator and attribute contents and cannot be reused for
another artifact. Low-level callers can pass the result of
aind_zarr_utils.io.ngff.interpreted_zarr(...) as opened_zarr.
Historical registration frames remain separate: pipeline images, stubs, and CCF point transforms keep the geometry used by registration. Physical pyramid translation correction does not modify the stored ANTs transform chain.
Pipeline majors 3 through 6 use the verified individual SyN warp/affine chain. The recorded registration-frame anchor is verified through importer 0.0.37; newer importer versions still emit a warning when that convention is assumed.
Earlier releases inferred a block-mean offset for every pyramid, regardless of stored translations. Callers relying on that behavior should now apply an evidenced correction explicitly. Synthetic pyramids should declare their actual translations in NGFF metadata.
Asset constructors use anonymous S3 access for metadata and image data by default.
Pass anonymous=False to use the normal AWS credential chain. Access settings
remain attached to each opened store and its child locations.
Images And Stubs
Asset.image() returns a SimpleITK image by default and can also return an ANTs
image. Asset.stub() returns a header-only SimpleITK image for coordinate
operations without loading pixel data.
from aind_zarr_utils import Asset, Origin
asset = Asset.from_root("s3://aind-open-data/dataset")
sitk_img = asset.image(level=3)
ants_img = asset.image(level=3, library="ants")
stub, size_ijk = asset.stub(level=0)
pipeline_stub, native_size_ijk = asset.stub(pipeline=True)
anchored = asset.image(
level=3,
origin=Origin.at_corner("RAS", (0.0, 0.0, 0.0)),
)
origin is only accepted when pipeline=False. Pipeline images and stubs use
the pipeline-corrected origin from processing.json.
Coordinate Spaces
Points stores named (N, 3) arrays plus a Space tag. Constructors validate
shape and coerce arrays to floating point.
import numpy as np
from aind_zarr_utils import Asset, Points, Space
asset = Asset.from_zarr("s3://bucket/asset/image.ome.zarr/0")
indices = Points(
{"soma": np.array([[100, 200, 50], [120, 180, 60]])},
Space.ZARR_INDICES,
)
pipeline_mm = asset.transform(indices, to=Space.LS_PIPELINE_ANATOMICAL_MM)
ccf_mm = asset.transform(indices, to=Space.CCF_MM)
round_trip = asset.transform(ccf_mm, to=Space.ZARR_INDICES)
Supported spaces are:
Space.ZARR_INDICES: continuous level-0(z, y, x)Zarr indicesSpace.LS_SCALED_MM: spacing-scaled light-sheet coordinatesSpace.LS_ANATOMICAL_MM: raw Zarr anatomical LPS millimetersSpace.LS_PIPELINE_ANATOMICAL_MM: pipeline-corrected LPS millimetersSpace.CCF_MM: Allen CCF LPS millimeters
SWC coordinates can enter the same graph without opening image data first:
swc_points = Points.from_swc(swc_array, axis_order="zyx", units="micrometer")
ccf_points = asset.transform(swc_points, to=Space.CCF_MM)
Legacy Functions
The lower-level modules remain available for compatibility:
aind_zarr_utils.zarr:zarr_to_ants,zarr_to_sitk,zarr_to_sitk_stubaind_zarr_utils.neuroglancer: Neuroglancer annotation readersaind_zarr_utils.pipeline_transformed: explicit metadata transform helpers
The auto-metadata convenience helpers in pipeline_transformed are deprecated
in favor of Asset.from_zarr(...) / Asset.from_root(...) plus
Asset.transform(...).
Installation
pip install aind-zarr-utils
For development:
git clone https://github.com/AllenNeuralDynamics/aind-zarr-utils.git
cd aind-zarr-utils
uv sync
Development
Run the core checks with:
uv run ruff format
uv run ruff check
uv run mypy
uv run pytest
uv run --group docs sphinx-build docs/source docs/build/html -W --keep-going
Pull requests use Angular-style commit messages:
<type>(<scope>): <short summary>
Metadata
Release files for aind-zarr-utils 0.20.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| aind_zarr_utils-0.20.0.tar.gz | 593.8 kB | Without provenance | Details for aind_zarr_utils-0.20.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| aind_zarr_utils-0.20.0-py3-none-any.whl | Python 3 | none | any | Details for aind_zarr_utils-0.20.0-py3-none-any.whl |
Total release size: 670.8 kB
Release files / aind_zarr_utils-0.20.0.tar.gz
| Download URL | aind_zarr_utils-0.20.0.tar.gz |
|---|---|
| Size | 593.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
2f36cbae7926d04aa35150cc47a70bcac75eba187dcb2258dbacb549d71afa5c
|
|
BLAKE2b-256 checksum How to use checksums |
968468e771bd0528062f677243962d14e1e62fcb82051b4aa6da391e79e68840
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.13.7 {"installer":{"name":"uv","version":"0.13.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / aind_zarr_utils-0.20.0-py3-none-any.whl
| Download URL | aind_zarr_utils-0.20.0-py3-none-any.whl |
|---|---|
| Size | 77.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
44a17ada5ef19736f86a22c2f730d37e6819ea9bf43b3a6abcf84b3f1adf229d
|
|
BLAKE2b-256 checksum How to use checksums |
f8e2fff25f6a2a0ba2715bb71a9b2dcb32c14015ff78aa66b20806bd16e0ad62
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.13.7 {"installer":{"name":"uv","version":"0.13.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Security for aind-zarr-utils 0.20.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.