EuBI-Bridge

EuBI-Bridge is a tool for distributed conversion of microscopy image collections to OME-Zarr. Conversion can be performed through either a command-line interface (CLI) or a graphical user interface (GUI). The installation described below provides access to both interfaces.

A key feature of EuBI-Bridge is aggregative conversion, which combines multiple input images along user-specified axes into a single OME-Zarr container. This is particularly useful when images belong to the same multidimensional dataset but are stored as separate, unlinked files. By aggregating these files during conversion, EuBI-Bridge can reconstruct the intended multidimensional dataset.

EuBI-Bridge is built on several powerful libraries, including zarr, tensorstore and dask, among others. OME-Zarr output is written by ome_zarr_pyramid on dyna-zarr, region by region within a memory budget.

Input files are read with micro-reader by default. It reads the common microscopy formats without Java. If a file cannot be read by micro-reader, EuBI-Bridge falls back to other readers (such as bioio- and Bio-Formats-based readers) for that particular file. Java only starts when this happens.

Installation

Important: EuBI-Bridge is currently only compatible with Python 3.11 or 3.12 due to conflicting dependencies. We are working on supporting a wider range of Python versions in future releases.

Recommended: conda

The recommended way to install EuBI-Bridge is in a conda environment. This route also installs Qt, so the graphical interface works without any further setup:

mamba create -n eubizarr openjdk=11.* maven python=3.12 pyqt6=6.8.1

Then install EuBI-Bridge via pip in the conda environment:

conda activate eubizarr
pip install --no-cache-dir eubi-bridge
eubi reset_config   # see "After installing or upgrading" below

Alternative: pip only

Installing into a plain virtual environment also works, provided your Python is version 3.11 or 3.12:

python -m venv venv # Python must be either version 3.11 or 3.12.
source venv/bin/activate
pip install eubi-bridge # installs both GUI and CLI
eubi reset_config   # see "After installing or upgrading" below

On Linux this installs the graphical interface but not the Qt system libraries it needs at runtime, so the graphical interface may fail to start. See Linux: the GUI does not start below for the fix. The command-line interface is unaffected.

Without Java: eubi-bridge-lite

eubi-bridge-lite is the same tool without Bio-Formats, Java and dask: a smaller install that works on any Python from 3.11. It converts the formats micro-reader reads; there is no Bio-Formats fallback for other files. The CLI and the GUI are both included.

pip install eubi-bridge-lite
eubi reset_config

After installing or upgrading

To upgrade an existing installation to the latest release:

pip install --upgrade eubi-bridge   # or: pip install --upgrade eubi-bridge-lite

Then, after installing or upgrading, run this once:

eubi reset_config

EuBI-Bridge keeps your settings in a configuration file. That file is written the first time you run the tool, and later versions do not change it. So if you have used EuBI-Bridge before, you keep the old defaults until you reset it.

• Resetting gives you the current defaults. For example, files are now read with micro-reader, and Java only starts if a file needs Bio-Formats.
• It overwrites your default settings. Run eubi show_config first if you want to note them down. Configurations you saved under a name are kept.
• On a fresh install it does no harm.

Some settings have new names. The old names still work.

• max_workers is now max_processes.
• max_concurrency is now max_threads.
• micro_read_concurrency is now decode_threads.
• bf_read_concurrency is now bioformats_threads.
• memory_per_worker is now slurm_worker_memory (SLURM only). On your own computer, use memory_budget (see Speed and memory).

Troubleshooting

Linux: the GUI does not start

This applies to the pip-only route. On Linux, pip installs PyQt6 but not the system libraries it needs at runtime, so the eubi-gui command (which is run to launch the GUI) may stop with an error such as:

EuBI-Bridge could not start its graphical interface: libEGL.so.1: cannot open shared object file

Install the Qt system libraries with your package manager:

# Debian / Ubuntu
sudo apt install libegl1 libgl1 libxkbcommon-x11-0 libdbus-1-3 libxcb-cursor0

# Fedora / RHEL
sudo dnf install mesa-libEGL mesa-libGL libxkbcommon-x11 dbus-libs xcb-util-cursor

If you cannot install system packages, use the recommended conda route instead: mamba create ... pyqt6=6.8.1 brings its own Qt libraries and avoids this problem entirely. The eubi command-line interface needs none of them and works either way.

Building wheel errors

If you receive a Building wheel error such as:

  Building wheel for ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1

then try the following:

# In the `eubizarr` environment
mamba install cmake zlib boost # preinstall dependencies that can help build from source
pip install --no-cache-dir eubi-bridge # try installing again with the dependencies available
eubi reset_config

Documentation

Find the documentation for EuBI-Bridge here

Graphical interface

Launch the graphical interface by running eubi-gui in the terminal. The four short videos below walk you through one complete job: converting several images to OME-Zarr, and then inspecting one of the outputs.

The videos cover the common scenario: one-to-one conversion from a collection of files. More advanced features such as aggregative conversions (concatenation of multiple files), editable batch tables, custom configuration files and editing output metadata are all supported by the interface but not yet demonstrated in the videos; demos for those will be provided soon. In the meantime the documentation describes the API, and every control in the interface carries a tooltip.

1. Selecting input and output folders

Navigate to your images, filter them with include and exclude patterns (which accept star * expressions), tick the files to be converted, and choose where the OME-Zarr output should be written.

https://github.com/user-attachments/assets/0a474606-d443-40f0-8f36-23053c61cc81

2. Choosing conversion parameters

Work through the parameter tabs. The tabs cover different parameter categories ranging from resource allocation, through reader options, chunking, sharding and compression, downscaling, channel and pixel metadata overrides, and more. Each field explains itself on hover, and settings can be saved to a configuration file for reuse. The Cluster tab sets how many images and threads run at once and how much memory the conversion may use (see Speed and memory).

https://github.com/user-attachments/assets/c7e3ab3b-37cf-4565-a37a-f2b6fa985961

3. Running a conversion

Start the job and watch it progress. The log reports each file as it is read, converted and downscaled, so a failure points at the file that caused it.

https://github.com/user-attachments/assets/3492e69d-ea83-4363-b3c7-90a9c00be76c

4. Inspecting the result

Switch to the Inspect tab to open one of the converted OME-Zarr stores from the list in the sidebar. In the Inspect/Metadata path, find full OME-Zarr metadata, including axis names, units, pixel sizes, the chunk and shard layout and compression per pyramid level. Pixel sizes and units can be corrected here and saved back into the OME-Zarr. Switch to the Inspect/Viewer path and visualise the OME-Zarr. Adjust a specific resolution layer via Zoom slider. Browse through the z slices or timepoints. Pan the frame of view using the left mouse button. Add or remove specific channels, and adjust the channel metadata such as colours, viewed pixel range and channel names. The updated visualisation metadata can also be saved back into the OME-Zarr.

https://github.com/user-attachments/assets/08149eb8-1cb6-403b-81a9-668ce4637f24

The same recordings are attached to the v0.1.2-media release for viewing outside GitHub. They are not part of the repository, so cloning or pip-installing the package does not download them.

Command line interface

Unary Conversion

Given a dataset structured as follows:

multichannel_timeseries
├── Channel1-T0001.tif
├── Channel1-T0002.tif
├── Channel1-T0003.tif
├── Channel1-T0004.tif
├── Channel2-T0001.tif
├── Channel2-T0002.tif
├── Channel2-T0003.tif
└── Channel2-T0004.tif

To convert each TIFF into a separate OME-Zarr container (unary conversion):

eubi to_zarr multichannel_timeseries multichannel_timeseries_zarr

By default this writes OME-Zarr 0.4 (zarr version 2). Use --ome_zarr_version to choose the version. For OME-Zarr 0.5 (zarr version 3, which supports sharding):

eubi to_zarr multichannel_timeseries multichannel_timeseries_zarr --ome_zarr_version 0.5

--zarr_format still works, but it is deprecated. Use --ome_zarr_version instead.

Both of these commands will perform unary conversion, resulting in the following output:

multichannel_timeseries_zarr
├── Channel1-T0001.zarr
├── Channel1-T0002.zarr
├── Channel1-T0003.zarr
├── Channel1-T0004.zarr
├── Channel2-T0001.zarr
├── Channel2-T0002.zarr
├── Channel2-T0003.zarr
└── Channel2-T0004.zarr

Use wildcards to specifically convert the images belonging to Channel1:

eubi to_zarr "multichannel_timeseries/Channel1*" multichannel_timeseries_channel1_zarr

Aggregative Conversion (Concatenation Along Dimensions)

To concatenate images along specific dimensions, EuBI-Bridge needs to be informed of file patterns that specify image dimensions. For this example, the file pattern for the channel dimension is Channel, which is followed by the channel index, and the file pattern for the time dimension is T, which is followed by the time index.

To concatenate along the time dimension:

eubi to_zarr multichannel_timeseries multichannel_timeseries_concat_zarr \
--channel_tag Channel \
--time_tag T \
--concatenation_axes t

Output:

multichannel_timeseries_concat_zarr
├── Channel1-T_tset.zarr
└── Channel2-T_tset.zarr

Important note: if the --channel_tag was not provided, the tool would not be aware of the multiple channels in the image and try to concatenate all images into a single one-channeled OME-Zarr. Therefore, when an aggregative conversion is performed, all dimensions existing in the input files must be specified via their respective tags.

For multidimensional concatenation (channel + time):

eubi to_zarr multichannel_timeseries multichannel_timeseries_concat_zarr \
--channel_tag Channel \
--time_tag T \
--concatenation_axes ct

Note that both axes are specified via the argument --concatenation_axes ct.

Output:

multichannel_timeseries_concat_zarr
└── Channel_cset-T_tset.zarr

Handling Nested Directories

For datasets stored in nested directories such as:

multichannel_timeseries_nested
├── Channel1
│   ├── T0001.tif
│   ├── T0002.tif
│   ├── T0003.tif
│   ├── T0004.tif
├── Channel2
│   ├── T0001.tif
│   ├── T0002.tif
│   ├── T0003.tif
│   ├── T0004.tif

EuBI-Bridge automatically detects the nested structure. To concatenate along both channel and time dimensions:

eubi to_zarr \
multichannel_timeseries_nested \
multichannel_timeseries_nested_concat_zarr \
--channel_tag Channel \
--time_tag T \
--concatenation_axes ct

Output:

multichannel_timeseries_nested_concat_zarr
└── Channel_cset-T_tset.zarr

To concatenate along the channel dimension only:

eubi to_zarr \
multichannel_timeseries_nested \
multichannel_timeseries_nested_concat_zarr \
--channel_tag Channel \
--time_tag T \
--concatenation_axes c

Output:

multichannel_timeseries_nested_concat_zarr
├── Channel_cset-T0001.zarr
├── Channel_cset-T0002.zarr
├── Channel_cset-T0003.zarr
└── Channel_cset-T0004.zarr

Selective Data Conversion

To recursively select specific files for conversion, wildcard patterns can be used. For example, to concatenate only timepoint 3 along the channel dimension:

eubi to_zarr \
"multichannel_timeseries_nested/**/*T0003*" \
multichannel_timeseries_nested_concat_zarr \
--channel_tag Channel \
--time_tag T \
--concatenation_axes c

Output:

multichannel_timeseries_nested_concat_zarr
└── Channel_cset-T0003.zarr

Note: When using wildcards, the input directory path must be enclosed in quotes as shown in the example above.

Handling Categorical Dimension Patterns

For datasets where channel names are categorical such as in:

blueredchannels_timeseries
├── Blue-T0001.tif
├── Blue-T0002.tif
├── Blue-T0003.tif
├── Blue-T0004.tif
├── Red-T0001.tif
├── Red-T0002.tif
├── Red-T0003.tif
└── Red-T0004.tif

Specify categorical names as a comma-separated list:

eubi to_zarr \
blueredchannels_timeseries \
blueredchannels_timeseries_concat_zarr \
--channel_tag Blue,Red \
--time_tag T \
--concatenation_axes ct

Output:

blueredchannels_timeseries_concat_zarr
└── BlueRed_cset-T_tset.zarr

Note that the categorical names are aggregated in the output OME-Zarr name.

With nested input structure such as in:

blueredchannels_timeseries_nested
├── Blue
│   ├── T0001.tif
│   ├── T0002.tif
│   ├── T0003.tif
│   └── T0004.tif
└── Red
    ├── T0001.tif
    ├── T0002.tif
    ├── T0003.tif
    └── T0004.tif

One can run the exact same command:

eubi to_zarr \
blueredchannels_timeseries_nested \
blueredchannels_timeseries_nested_concat_zarr \
--channel_tag Blue,Red \
--time_tag T \
--concatenation_axes ct

Output:

blueredchannels_timeseries_nested_concat_zarr
└── BlueRed_cset-T_tset.zarr

Speed and memory

EuBI-Bridge reads each image once and writes all resolution levels from that read. It stays within the memory you give it, however large the images are.

You set three things:

• --max_processes: how many images are converted at once, each in its own process.
• --max_threads: how many parts of an image each conversion works on at once.
• --memory_budget: how much memory the whole run may use, for example 16GB. The default, auto, is half of the memory that is free when the run starts.

eubi to_zarr input_dir output_dir --max_processes 4 --max_threads 4 --memory_budget 8GB

For example, 2347 TIFF planes (17.6 GB) were converted into a 6-level OME-Zarr 0.5 with large shards in about 1.3 minutes with --memory_budget 1.5GB, using less than 1 GB.

Set them once with eubi configure cluster, or in the Cluster tab of the graphical interface. On a SLURM cluster, --slurm_worker_memory sets the memory of each worker job instead.

The OME-Zarr is written by ome_zarr_pyramid by default. If you need to, you can choose:

• --writer eubi: EuBI-Bridge's own writer. It gives the same pixels.
• --io_backend zarrista: write with the Rust-based zarrista library. Install it with pip install "dyna-zarr[zarrista]".

Additional Notes

• EuBI-Bridge is in the beta stage, and significant updates may be expected.
• Community support: Questions and contributions are welcome! Please report any issues.