BBBC Segmentation Datasets (2D/3D) - A PyTorch Dataset Manager

📌 Overview

This package provides a PyTorch-compatible dataset manager for Broad Bioimage Benchmark Collection (BBBC) datasets.

It allows users to:

• Download and manage BBBC segmentation datasets automatically
• Load datasets as PyTorch Dataset objects
• Handle 2D & 3D images properly
• Support image augmentations via torchvision.transforms (#TODO)

🚀 Installation

pip install bbbc-datasets

git clone https://github.com/mario-koddenbrock/bbbc_datasets.git
cd bbbc_datasets

pip install -r requirements.txt

📂 Available Datasets

You can list all available BBBC datasets without downloading them. You can also filter the datasets to show only 2D, only 3D, or both types of datasets.

from examples.load_dataset import list_available_datasets

# List all datasets
list_available_datasets()

# List only 2D datasets
list_available_datasets(filter_3d=False)

# List only 3D datasets
list_available_datasets(filter_3d=True)

📂 Available BBBC Datasets:
- BBBC001: Human HT29 colon-cancer cells for counting.
- BBBC002: Drosophila Kc167 cells for counting.
- BBBC003: DIC microscopy images of mouse embryos for segmentation analysis.
- BBBC004: Synthetic fluorescence microscopy images of nuclei with varying clustering probabilities.
- BBBC005: Simulated fluorescence microscopy images with varying focus blur.
- BBBC006: Z-stack fluorescence microscopy images of human U2OS cells.
- BBBC007: Drosophila Kc167 cells for cell boundary segmentation.
- BBBC008: Dataset: Fluorescence microscopy images of human HT29 colon cancer cells.
- BBBC009: Human red blood cells (DIC images).
- BBBC010: Live/dead assay of C. elegans exposed to pathogens.
- BBBC018: Human HT29 colon-cancer cells (diverse phenotypes).
- BBBC019: Collective cell migration (wound healing, DIC images).
- BBBC020: Murine bone-marrow derived macrophages (overlapping cells).
- BBBC024: 3D Synthetic HL60 Cell Line Images with Varying Clustering Probability and SNR.
- BBBC027: 3D Synthetic Colon Tissue Images with Varying SNR.
- BBBC028: Polymerized Structures in Differential Interference Contrast (DIC) Microscopy.
- BBBC029: Synthetic Differential Interference Contrast (DIC) Images.
- BBBC030: Chinese Hamster Ovary (CHO) cells (DIC images).
- BBBC032: 3D Mouse Embryo Blastocyst Cells.
- BBBC033: 3D Mouse Trophoblast Stem Cells.
- BBBC034: 3D Induced Pluripotent Human Stem Cells (hiPSC).
- BBBC035: Simulated HL60 Cells from the Cell Tracking Challenge.
- BBBC038: Kaggle 2018 Data Science Bowl - Nuclei Segmentation.
- BBBC039: Nuclei of U2OS Cells in a Chemical Screen.
- BBBC044: Mouse hippocampal presynaptic terminals (vGlut-1 and vGat).
- BBBC046: FiloData3D - Synthetic 3D Time-Lapse Imaging of A549 Cells with Filopodia.
- BBBC050: Nuclei of Mouse Embryonic Cells.

Filtering by metadata

Every dataset carries a fixed-schema DatasetMetadata object (the METADATA class attribute, see bbbc_datasets/metadata.py) describing its imaging setup and content. Unknown values are None — never guessed.

from bbbc_datasets.dataset_manager import DatasetManager

# Metadata of a single dataset
meta = DatasetManager.get_metadata("BBBC044")
meta.dim        # "3D"
meta.to_dict()  # plain dict (JSON-serializable)

# Filter by any metadata field (no downloads, no instantiation)
DatasetManager.filter_datasets(dim="3D")            # all 3D datasets
DatasetManager.filter_datasets(data_type="synthetic")
DatasetManager.filter_datasets(imaging_mode="dic", label_type="foreground")
DatasetManager.filter_datasets(task="counting")
# legacy: filter_3d=True/False is equivalent to dim="3D"/"2D"

Fields: name, title, source, dim (2D/3D), imaging_mode (fluorescence/dic/brightfield/brightfield_fluorescence), modality (grayscale/rgb/multichannel), resolution_um (tuple, None when not uniform), image_size ((H, W) or (Z, H, W), None when varying), data_type (synthetic/real), instrument, n_images, task (segmentation/counting/detection/classification), label_type (foreground/instance/outlines/counts/bounding_boxes/biological), doi, license.

Dataset previews

Each dataset below is a collapsible entry with three downscaled example images (the label mask is shown side by side when available) and the full metadata. The thumbnails live in gallery/ and are generated from the downloaded data:

from bbbc_datasets.dataset_manager import DatasetManager

# Regenerate the thumbnails (requires the datasets to be downloaded)
DatasetManager.gallery("gallery", examples=3, max_size=300)
# or: python -m bbbc_datasets.gallery --output gallery

Note: some sets are named "3D" by the BBBC but this package processes single 2D frames of them (BBBC032–BBBC035, BBBC046); their metadata dim is therefore 2D, matching the code's is_3d flag.

🎨 Visualizing Dataset Samples

Display First Image from Each Dataset

Requires downloading datasets first!

You can display samples from the datasets. Similar to listing datasets, you can filter the samples to show only 2D, only 3D, or both types of datasets.

from examples.load_dataset import display_dataset_samples

# Display samples from all datasets
display_dataset_samples()

# Display samples from only 2D datasets
display_dataset_samples(filter_3d=False)

# Display samples from only 3D datasets
display_dataset_samples(filter_3d=True)

📥 Download and Load a Dataset

Load a Dataset as a PyTorch Dataset

from bbbc_datasets.datasets.bbbc004 import BBBC004
from torch.utils.data import DataLoader

# Load BBBC004 dataset
dataset = BBBC004()

# Create a DataLoader
dataloader = DataLoader(dataset, batch_size=4, shuffle=True)

# Iterate through data
for images, labels in dataloader:
    print(f"Batch shape: {images.shape}, Labels: {labels.shape if labels is not None else 'None'}")

DatasetManager.filter_datasets filters the dataset classes by metadata without triggering any downloads (see Filtering by metadata):

from bbbc_datasets.dataset_manager import DatasetManager

# All datasets
datasets_all = DatasetManager.filter_datasets()

# All 3D datasets (legacy filter_3d still works)
datasets_3d = DatasetManager.filter_datasets(filter_3d=True)

# Or filter by any metadata field
datasets_dic = DatasetManager.filter_datasets(imaging_mode="dic")

Dataset-Specific Options

Some datasets ship in several variants and can be loaded selectively:

# Only one of the eight wound-healing subsets of BBBC019
dataset = BBBC019(subset="MDCK")

# Only the vGat marker of the 3D BBBC044 stacks
dataset = BBBC044(marker="vgat")

🛠 Running Tests

python -m unittest tests/test_urls.py

python -m unittest tests/test_dataset_loading.py

📝 Up-next

• Complete the non-BBBC stubs: cellpose and livecell are partial, sartorius needs its class body fixed, and evican/isbi15 are empty placeholders.
• Cell Painting profiling sets (BBBC025/036/047): multi-channel image-profiling data, a different paradigm from the current segmentation focus.
• Biological-label sets (BBBC041/042/045): only bounding boxes / biological labels, no segmentation masks yet.

🆕 v1.0.0 highlights

• 9 new BBBC datasets: BBBC001, BBBC002, BBBC007, BBBC009, BBBC018, BBBC019, BBBC020, BBBC030, BBBC044 (now 27 total).
• Robust downloading: per-file markers so multi-archive datasets (e.g. the 8 BBBC019 subsets) no longer skip files; metadata files are downloaded into a metadata/ folder.
• Correct image/label discovery: deep recursive listing, __MACOSX junk filtering, and exact stem matching before fuzzy matching (fixes BBBC039 label mismatches).
• Working PyTorch wrapper: DatasetManager.get_dataset / DataLoader now return correct per-sample (image, label) tensors for 2D, 3D, RGB and ArrayScan .DIB inputs.

🙌 Contributing

Contributions are welcome! If you find issues or want to add new features, feel free to:

• Submit a Pull Request
• Report Issues in the GitHub Issues section

📜 License

This project is licensed under the MIT License.