Documentation available here. 📚

Contents: Intro • Quickstart • Napari plugin • Installation • Recent news • Browser usage • Limitations • Roadmap • Related tools • Contributing • Citing • License

multiview-stitcher

multiview-stitcher is an open-source modular toolbox for distributed and tiled stitching of 2-3D image data in Python. It is a collection of algorithms to register and fuse small and large datasets from multi-positioning and multi-view light sheet microscopy, as well as other modalities such as correlative cryo-EM datasets. As such, it shares considerable functionality with the Fiji plugin BigStitcher, with the difference that it is designed for interoperability with the Python scientific ecosystem. This allows it to:

• easily integrate into existing Python-based workflows (within Jupyter notebooks, scripts, etc.) 🐍
• scale to very large datasets using mature Python tooling (using dask, zarr-python, ray) 🚀
• make use of community-developed data representations (xarray.DataArray, xarray.DataTree, spatialdata) 🤓
• ensure compatibility with and optimal usage of modern file formats and standards, e.g. OME-Zarr
• swap in custom methods for registration and fusion that are readily available in the Python ecosystem (e.g. from scikit-image, ANTs, elastix, SimpleITK) 🔧

👀 Visualization: The associated napari-stitcher provides visualization functionality using the Napari viewer, including a standalone widget for stitching vanilla napari image layers. Alternatively, web-based visualization of huge datasets together with their associated transformations is supported using neuroglancer (no additional installation required! See e.g. the exaSPIM example notebook).

🛠️ Extensibility: Next to the built-in functions for pairwise registration, fusion and view weighing, custom functions with a simple API can be provided by the user. Multiview-stitcher provides these functions with chunk-sized and pre-transformed image arrays, taking care of the overall stitching workflow and large data handling.

🚀 Scalability: The package is designed to handle very large datasets that do not fit into memory. It leverages zarr, dask and ray for efficient data handling and processing. For example, multiview-stitcher can fuse cloud-hosted exaSPIM datasets of >100TB each (see example notebook).

🔄 Transformations: multiview-stitcher supports both input and output tile transformations, as well as registration results to be full affine transformations. This includes simple shifts / translations, as well as rotation and scaling for advanced stitching or multi-view fusion. Non-rigid transformations are not supported at the moment.

Quickstart

• Documentation and code example
• Check out the example notebooks.

Code example

These code snippets walk you through a small stitching workflow consisting of

1. Preparing the input image data and metadata (tile positions, spacing, channels)
2. Registering the tiles
3. Stitching / fusing the tiles

1) Prepare data for stitching

import numpy as np
from multiview_stitcher import msi_utils
from multiview_stitcher import spatial_image_utils as si_utils

# input data (can be any numpy compatible array: numpy, dask, cupy, etc.)
tile_arrays = [np.random.randint(0, 100, (2, 10, 100, 100)) for _ in range(3)]

# indicate the tile offsets and spacing
tile_translations = [
    {"z": 2.5, "y": -10, "x": 30},
    {"z": 2.5, "y": 30, "x": 10},
    {"z": 2.5, "y": 30, "x": 50},
]
spacing = {"z": 2, "y": 0.5, "x": 0.5}

channels = ["DAPI", "GFP"]

# build input for stitching
msims = []
for tile_array, tile_translation in zip(tile_arrays, tile_translations):
    sim = si_utils.get_sim_from_array(
        tile_array,
        dims=["c", "z", "y", "x"],
        scale=spacing,
        translation=tile_translation,
        # affine=None, # (e.g. for rotated or sheared tiles)
        transform_key="stage_metadata",
        c_coords=channels,
    )
    msims.append(msi_utils.get_msim_from_sim(sim, scale_factors=[]))

# plot the tile configuration
# from multiview_stitcher import vis_utils
# fig, ax = vis_utils.plot_positions(msims, transform_key='stage_metadata', use_positional_colors=False)

2) Register the tiles

from dask.diagnostics import ProgressBar
from multiview_stitcher import registration

with ProgressBar():
    params = registration.register(
        msims,
        reg_channel="DAPI",  # channel to use for registration
        transform_key="stage_metadata",
        new_transform_key="translation_registered",
        pre_registration_pruning_method=None,
        plot_summary=True,
    )

3) Stitch / fuse the tiles

from multiview_stitcher import fusion

fused_msim = fusion.fuse(
    images=msims,
    transform_key="translation_registered",
)

# get fused array at the highest output resolution as a dask array
fused_msim["scale0/image"].data

# get fused array at the highest output resolution as a numpy array
fused_msim["scale0/image"].data.compute()

Because the input is a list of MultiscaleSpatialImage objects, lazy fusion returns a fused MultiscaleSpatialImage.

For large datasets (>50GB, potentially with benefits already at >5GB) consider streaming the fused result directly to a zarr file using the following way to call fusion.fuse:

from multiview_stitcher import fusion

fused_sim = fusion.fuse(
    images=msims,
    transform_key="translation_registered",
    # ... further optional args for fusion.fuse
    output_zarr_url="fused_output.ome.zarr",
    zarr_options={
        "ome_zarr": True,
        # "ngff_version": "0.4",  # optional
    },
    # optionally, we can use joblib for parallelization (`pip install joblib` and `from multiview_stitcher import misc_utils`):
    # batch_options={
    #     "batch_func": misc_utils.process_batch_using_joblib,
    #     "n_batch": 20,  # number of chunk fusions to schedule / submit at a time
    #     "batch_func_kwargs": {
    #         'n_jobs': 4  # number of parallel jobs for joblib
    #     },
    # },
)

Napari plugin

There's an associated napari plugin: napari-stitcher.

Image data by Arthur Michaut @ Jérôme Gros Lab @ Institut Pasteur.

Installation

You can install multiview-stitcher via pip from PyPI:

pip install multiview-stitcher

or from the source code in this github repository:

pip install git+https://github.com/multiview-stitcher/multiview-stitcher.git

Recent news

• Oct/25 (v0.1.37): Support for fusing huge datasets using fusion.fuse(..., output_zarr_url=...), in which the fused result is streamed to disk in batches of independently processed chunks, circumventing any dask graph induced overhead. Tested on >100TB datasets!
• Oct/25 (v0.1.34): register(..., reg_res_level=1) for registering directly on downsampled data
• Aug/25 (v0.1.30): Multi-view fusion example notebook available.
• May/25 (v0.1.26): Introduced option to specify the number of parallel pairwise registrations for improved performance / memory tradeoff.
• Mar/25 (v0.1.23): Support for neuroglancer visualization of
  • input tiles together with their input transformations
  • registered tiles together with their registration transformations
  • fused output together with the transformations of all input tiles
• Mar/25 (v0.1.21): Obtained completely stable numerics for n-dimensional stack intersection calculation using scipy.spatial.HalfspaceIntersection.

Citing multiview-stitcher

If you find multiview-stitcher useful please cite this repository using the following DOI (all versions): https://doi.org/10.5281/zenodo.13151252.

Stitching in the browser

multiview-stitcher can run without installation in your browser. Data is processed locally in the browser and not uploaded to any server.

Web app

A browser app combines the library with neuroglancer as the viewer:

• drop a folder holding your OME-Zarr tiles onto the page (Chromium-based browsers, using the File System Access API)
• register and fuse with a single click, spread over as many Python web workers as you choose
• switch the displayed transform_key to compare coordinate systems, e.g. stage positions vs. registration result
• fuse either lazily into the viewer, or into an OME-Zarr written back to a folder on your disk

See Stitching in the browser for how it works and its current limitations.

JupyterLite

• open JupyterLite in a private browser window
• upload this notebook into the jupyter lab window: notebooks/stitching_in_the_browser.ipynb
• upload files to stitch into a 'data' folder in the jupyter lab window
• follow the notebook

Limitations: stitching will run with a single thread and while the code runs locally, your local file system is not directly accessible from within the browser environment

Known limitations

1. The current implementation focuses on rigid transformations (translation, rotation). Non-rigid transformations are not supported at the moment.
2. In terms of data volumes, processing huge tiles is handled well. A large amount of tiles (e.g. more than hundreds) works but can be slow during registration, as the currently built-in global optimization method converges slowly for large numbers of tiles.
3. Open an issue if you encounter any problems or have suggestions for improvements 🙋

Roadmap / Future plans

Some planned improvements for future releases:

1. Implement a hierarchical and parallelised global registration optimization for faster registration of datasets with large numbers of tiles (>100s).
2. Implement more built-in registration and fusion methods:
   1. Feature-based registration
   2. Multiview deconvolution-based fusion
3. The built-in option to subdivide tiles / views for working with piecewise affine transformations that account for local distortions observed in e.g. large FOV light sheet data.
4. Make multiview-stitcher available via conda-forge.
5. Open an issue if you have suggestions for improvements 🙋

Related stitching tools

multiview-stitcher sits in a broader ecosystem of excellent open-source stitching software.
Rather than aiming to replace existing tools, it focuses on providing a Python-native, modular API that integrates well with the scientific Python ecosystem (dask/zarr/xarray, napari, etc.).

The table below is a high-level orientation (features and workflows often overlap, and most tools can be combined in practice):

* “Out-of-core / huge data” depends heavily on workflow, file formats, and output options. Several tools here can handle very large datasets; in practice, Python can make it particularly convenient to compose and distribute stitching workflows across compute resources.

Rule of thumb

• If you want a mature GUI-first workflow with broad microscopy stitching functionality: BigStitcher
• If you stitch whole-slide multiplexed 2D mosaics and want a simple CLI: Ashlar
• If you stitch very large 3D tiled volumes with a dedicated toolchain: TeraStitcher
• If you want stitching and fusion as a Python building block that plugs into existing analysis pipelines: multiview-stitcher

If you spot inaccuracies or want to extend the comparison, please open an issue or PR 🙂

Work in progress

multiview-stitcher is being actively developed in the open and the API is subject to change.

Previous work

multiview-stitcher improves and replaces MVRegFUS.

Issues

If you encounter any problems, please file an issue along with a description of the problem. Interacting with the community and developers via issues is highly appreciated and encouraged 🙌

Contributing

Contributions are very welcome 🙌

If you're looking for ideas, feel free to have a look at the open issues (e.g. those labeled with "help wanted" or "good first issue").

License

Distributed under the terms of the BSD-3 license, "multiview-stitcher" is free and open source software.