bio-volumentations 1.3.0

Library for 3D-5D augmentations of volumetric multi-dimensional biomedical images and their annotations

Bio-Volumentations

Bio-Volumentations is an image augmentation and preprocessing package for 3D (volumetric), 4D (time-lapse volumetric or multi-channel volumetric), and 5D (time-lapse multi-channel volumetric) biomedical images and their annotations.

The library offers a wide range of efficiently implemented image transformations. This includes both preprocessing transformations (such as intensity normalisation, padding, and type casting) and augmentation transformations (such as affine transform, noise addition and removal, and contrast manipulation).

The Bio-Volumentations library is a suitable tool for image manipulation in machine learning applications. It can transform several types of reference annotations along with the image data and it can be used with any major Python deep learning library, including PyTorch, PyTorch Lightning, TensorFlow, and Keras.

This library builds upon wide-spread libraries such as Albumentations and TorchIO (see the Contributions section below). Therefore, it can easily be adopted by developers.

Installation

Install the package from pip using:

pip install bio-volumentations

See the project's PyPI page for more details.

Requirements

• NumPy
• SciPy
• Scikit-image
• SimpleITK

Usage

Importing

Import the library to your project using:

import bio_volumentations as biovol

How to Use Bio-Volumentations?

The Bio-Volumentations library processes 3D, 4D, and 5D images. Each image must be represented as a numpy.ndarray and must conform to the following conventions:

• The order of dimensions is [C, Z, Y, X, T], where C is the channel dimension, T is the time dimension, and Z, Y, and X are the spatial dimensions.
• The three spatial dimensions (Z, Y, X) must be present. To transform a 2D image, please create a dummy Z dimension.
• The channel (C) dimension is optional. If it is not present, the library will automatically create a dummy dimension in its place, so the output image shape will be [1, Z, Y, X].
• The time (T) dimension is optional and can only be present if the channel (C) dimension is also present in the input data. To process single-channel time-lapse images, please create a dummy C dimension.

Thus, an input image is interpreted in the following ways based on its dimensionality:

1. 3D: a single-channel volumetric image [Z, Y, X];
2. 4D: a multi-channel volumetric image [C, Z, Y, X];
3. 5D: a single- or multi-channel volumetric image sequence [C, Z, Y, X, T].

The shape of the output image is either [C, Z, Y, X] (cases 1 & 2) or [C, Z, Y, X, T] (case 3).

The images are type-casted to a floating-point datatype before being transformed, irrespective of their actual datatype.

For the specification of image annotation conventions, please see below.

The transformations are implemented as callable classes inheriting from an abstract Transform class. Upon instantiating a transformation object, one has to specify the parameters of the transformation.

All transformations work in a fully 3D fashion. Individual channels and time points of a data volume are usually transformed separately and in the same manner; however, certain transformations can also work along these dimensions. For instance, GaussianBlur can perform the blurring along the temporal dimension and with different strength in individual channels.

The data can be transformed by a call to the transformation object. It is strongly recommended to use Compose to create and use transformation pipelines.
An instantiated Compose object encapsulates the full transformation pipeline and provides additional support: it automatically checks and adjusts image format and datatype, outputs the image as a contiguous array, and can optionally convert the transformed image to a desired format. If you call transformations outside of Compose, we cannot guarantee the all assumptions are checked and enforced, so you might encounter unexpected behaviour.

Below, there are several examples of how to use this library. You are also welcome to check our documentation pages.

Example: Transforming a Single Image

To create the transformation pipeline, you just need to instantiate all desired transformations (with the desired parameter values) and then feed a list of these transformations into a new Compose object.

Optionally, you can specify a datatype conversion transformation that will be applied after the last transformation in the list, e.g. from the default numpy.ndarray to a torch.Tensor. You can also specify the probability of actually applying the whole pipeline as a number between 0 and 1. The default probability is 1 - the pipeline is applied for each call. See the docs for more details.

The Compose object is callable. The data is passed as a keyword argument, and the call returns a dictionary with the same keywords and corresponding transformed images. This might look like an overkill for a single image, but will come handy when transforming images with annotations. The default key for the image is 'image'.

import numpy as np
from bio_volumentations import Compose, RandomGamma, RandomRotate90, GaussianBlur

# Create the transformation pipeline using Compose
aug = Compose([
        RandomGamma(gamma_limit = (0.8, 1.2), p = 0.8),
        RandomRotate90(axes = [1, 2, 3], p = 1),
        GaussianBlur(sigma = 1.2, p = 0.8)
      ])

# Generate an image - shape [C, Z, Y, X]
img = np.random.rand(1, 128, 256, 256)

# Transform the image
# Please note that the image must be passed as a keyword argument to the transformation pipeline
# and extracted from the outputted dictionary.
data = {'image': img}
aug_data = aug(**data)
transformed_img = aug_data['image']

Example: Transforming Images with Annotations

Sometimes, it is necessary to transform an image with some corresponding additional targets. To that end, Bio-Volumentations define several target types:

• image for the image data;
• mask for integer-valued label images;
• float_mask for real-valued label images;
• keypoints for a list of key points; and
• value for non-transformed values.

For more information on the format of individual target types, see the Getting Started guide

If a Random... transform receives multiple targets on its input in a single call, the same transformation parameters are used to transform all of these targets. For example, RandomAffineTransform applies the same geometric transformation to all target types in a single call.

Some transformations, such as RandomGaussianNoise or RandomGamma, are only defined for the image target and leave the mask and float_mask targets unchanged. Please consult the documentation of the individual transforms for more details.

The corresponding targets are fed to the Compose object call as keyword arguments and extracted from the outputted dictionary using the same keys. The default key values are 'image', 'mask', 'float_mask', 'keypoints', and 'value'.

import numpy as np
from bio_volumentations import Compose, RandomGamma, RandomRotate90, GaussianBlur

# Create the transformation using Compose
aug = Compose([
        RandomGamma(gamma_limit = (0.8, 1.2), p = 0.8),
        RandomRotate90(axes = [1, 2, 3], p = 1),
        GaussianBlur(sigma = 1.2, p = 0.8)
      ])

# Generate image and a corresponding labeled image
img = np.random.rand(1, 128, 256, 256)
lbl = np.random.randint(0, 1, size=(128, 256, 256), dtype=np.uint8)

# Transform the images
# Please note that the images must be passed as keyword arguments to the transformation pipeline
# and extracted from the outputted dictionary.
data = {'image': img, 'mask': lbl}
aug_data = aug(**data)
transformed_img, transformed_lbl = aug_data['image'], aug_data['mask']

Example: Transforming Multiple Images of the Same Target Type

You can input arbitrary number of inputs to any transformation. To achieve this, you have to define the keywords for the individual inputs when creating the Compose object. The specified keywords will then be used to input the images to the transformation call as well as to extract the transformed images from the outputted dictionary.

Specifically, you can define image-type target keywords using the img_keywords parameter - its value must be a tuple of strings, each string representing a single keyword. Similarly, there are mask_keywords, fmask_keywords, value_keywords, and keypoints_keywords parameters for the respective target types.

Importantly, there must always be an image-type target with the keyword 'image'. Otherwise, the keywords can be any valid dictionary keys, and they must be unique within each target type.

You do not need to use all specified keywords in a transformation call. However, at least the target with the 'image' keyword must be present in each transformation call. In our example below, we only transform three targets even though we defined four target keywords explicitly (and there are some implicit keywords as well for the other target types).

You cannot define your own target types; that would require re-implementing all existing transforms.

import numpy as np
from bio_volumentations import Compose, RandomGamma, RandomRotate90, GaussianBlur

# Create the transformation using Compose: do not forget to define targets
aug = Compose([
        RandomGamma(gamma_limit = (0.8, 1.2), p = 0.8),
        RandomRotate90(axes = [1, 2, 3], p = 1),
        GaussianBlur(sigma = 1.2, p = 0.8)
    ],
    img_keywords=('image', 'abc'), mask_keywords=('mask',), fmask_keywords=('nothing',))

# Generate the image data: two images and a single int-valued mask
img = np.random.rand(1, 128, 256, 256)
img1 = np.random.rand(1, 128, 256, 256)
lbl = np.random.randint(0, 1, size=(128, 256, 256), dtype=np.uint8)

# Transform the images
# Please note that the images must be passed as keyword arguments to the transformation pipeline
# and extracted from the outputted dictionary.
data = {'image': img, 'abc': img1, 'mask': lbl}
aug_data = aug(**data)
transformed_img = aug_data['image']
transformed_img1 = aug_data['abc']
transformed_lbl = aug_data['mask']

Example: Adding a Custom Transformation

Each transformation inherits from the Transform class. You can thus easily implement your own transformations and use them with this library. You can check our implementations to see how this can be done. For example, Flip can be implemented as follows:

import numpy as np
from typing import List
from bio_volumentations import DualTransform

class Flip(DualTransform):
    def __init__(self, axes: List[int] = None, always_apply=False, p=1):
        super().__init__(always_apply, p)
        self.axes = axes

    # Transform the image
    def apply(self, img, **params):
        return np.flip(img, params["axes"])

    # Transform the int-valued mask
    def apply_to_mask(self, mask, **params):
       # The mask has no channels
        return np.flip(mask, axis=[item - 1 for item in params["axes"]])
    
    # Transform the float-valued mask
    # By default, float_mask uses the implementation of mask, unless it is overridden (see the implementation of DualTransform).
    #def apply_to_float_mask(self, float_mask, **params):
    #    return self.apply_to_mask(float_mask, **params)

    # Get transformation parameters. Useful especially for RandomXXX transforms to ensure consistent transformation of image tuples.
    def get_params(self, **data):
        axes = self.axes if self.axes is not None else [1, 2, 3]
        return {"axes": axes}

Implemented Transforms

A List of Implemented Transformations

Point transformations:

GaussianNoise 
PoissonNoise
RandomGamma 
RandomBrightnessContrast 
HistogramEqualization 
Normalize
NormalizeMeanStd

Local transformations:

GaussianBlur 
RandomGaussianBlur

Geometrical transformations:

AffineTransform
Resize 
Scale
Flip 
CenterCrop 
Pad
RandomAffineTransform
RandomScale 
RandomRotate90
RandomFlip 
RandomCrop

Contributions

Authors of the Bio-Volumentations library: Samuel Šuľan, Lucia Hradecká, Filip Lux.

• Lucia Hradecká: [e-mail]
• Filip Lux: [e-mail]

The Bio-Volumentations library is based on the following image augmentation libraries:

• Albumentations
• Volumentations
• Volumentations: Continued Development
• Volumentations: Enhancements
• Volumentations: Further Enhancements
• TorchIO

We would thus like to thank their authors, namely the Albumentations team, Pavel Iakubovskii, ZFTurbo, ashawkey, Dominik Müller, and TorchIO contributors.

Citation

TBA