Snakemake workflow: dvp-imaging-pipeline

A Snakemake workflow for Scalable processing of DVP imaging data with snakemake.

• Snakemake workflow: dvp-imaging-pipeline
  • Scope
  • Usage
  • References
  • TODO

About

Workflow

flowchart LR


style QCTissue1 fill:#ffffff,stroke:#222222,stroke-width:2px,color:#000
style QCSegmentation1 fill:#ffffff,stroke:#222222,stroke-width:2px,color:#000
style QCClassification1 fill:#ffffff,stroke:#222222,stroke-width:2px,color:#000
style QCClassification2 fill:#ffffff,stroke:#222222,stroke-width:2px,color:#000

%% colors
%% DONE: #0097A7
style IO fill:#ffffff,stroke:#0097A7,stroke-width:4px,color:#000
style Segmentation1a fill:#ffffff,stroke:#0097A7,stroke-width:2px,color:#000
style Classification1a fill:#ffffff,stroke:#0097A7,stroke-width:2px,color:#000
style Classification2a fill:#ffffff,stroke:#0097A7,stroke-width:2px,color:#000




IO[Generate spatialdata]

Tissue1[Tissue Identification]
Tissue2[Dearraying]
Tissue3a[Tissue QC, IF]
Tissue3b[Tissue QC, HE]

Segmentation1a[Cell/Nuclei segmentation, IF]
Segmentation1b[Cell/Nuclei segmentation, HE]


Classification1a[QC, IF]
Classification2a[Cell Classification, IF]
Classification2b[Cell Classification, HE]

QCTissue1(Tissue Plots)
QCSegmentation1(Segmentation Plots + Metrics)
QCClassification1(QC Plots + Metrics)
QCClassification2(Classification Plots + Metrics)


IO --> Tissue1
Tissue1 -->|TMA| Tissue2
Tissue1 -->|WSI| Tissue3a
Tissue1 -->|WSI| Tissue3b

Tissue2 --> Tissue3a
Tissue2 --> Tissue3b


Tissue3a --> Segmentation1a
Tissue3b --> Segmentation1b

Segmentation1a --> Classification1a --> Classification2a
Segmentation1b --> Classification2b

Scope

Priority		Current implementation/status
Must have
	Spatialdata IO	:white_check_mark: (dvpio)
	Nuclei+Cell segmentation	:white_check_mark: (harpy)
	Align cells+nuclei	(harpy)
	Tissue QC	(GrandQC, custom)
	Cell QC	:white_check_mark: (custom)
	Cell classification	:white_check_mark: (GMM)
	Workflow manager (Snakemake)
	Documentation+Tutorials
Nice to have
	Automated Reports
	Spatialdata database
	Config editor
	Train cellpose models in spatialdata
	Dockerized solution
Not in scope
	Complete GUI

Usage

Detailed information about input data and workflow configuration can also be found in the config/README.md.

If you use this workflow in a paper, don't forget to give credits to the authors by citing the URL of this repository or its DOI.

Deployment options

To run the workflow from the command line, change the working directory.

From repository

Clone this repository or download it via (Code > Download ZIP)

git clone https://github.com/lucas-diedrich/dvp-imaging-pipeline.git

cd path/to/dvp-imaging-pipeline

Adjust options in the default config file config/config.yaml. Before running the complete workflow, you can perform a dry run using:

snakemake --dry-run

To run the workflow with test files using conda:

snakemake --logger snkmt --cores 2 --sdm conda --directory workflow --configfile tests/integration/config/config.yaml

Results (spatialdata objects + logs + plots) can be found in the /tests/results directory.

Logging

Monitor the progress of the workflow in the workflow directory with snkmt, see also the documentation

snkmt console

Install utility functions

The utility functions in the src/ directory are a viable python package (dvpimg) and can be installed with pip.

1. In the shell, go to your favorite directory. Create a suitable environment, e.g. with mamba

mamba create -n dvpimg python=3.12 -y && mamba activate dvpimg

2. Clone the repository

git clone https://github.com/lucas-diedrich/dvp-imaging-pipeline.git
cd dvp-imaging-pipeline

3. Install the package

pip install .

Docker

Todo

HPC

You can run this workflow on an high-performance computing cluster

On the cluster, create the snakemake base environment the environment

conda create -n snakemake -y
conda env update --n snakemake --file environment.yaml

Additionally install the snakemake-executor-plugin-slurm:

pip install snakemake-executor-plugin-slurm

Then submit the provided workflow script on a cluster. Please checkout the script and the official snakemake slurm plugin documentation to be learn about relevant flags and settings.

cd /workflow/
sbatch snakemake.sbatch

Tests

You can run an example workflow with hatch (it automatically manages the required dependencies)

# Install hatch
pip install hatch

# Install all dependencies+environments for package + snakemake workflow
hatch run workflow:install

# Dry run
hatch run workflow:dry-run

# Full test with example data
hatch run workflow:run-test

Components

See the documentation for more information on the individual components and detailed instructions on their configuration.

References

(alphabetical order)

dvp-io: MannLabs/dvp-io. Lucas Diedrich (2025). https://github.com/MannLabs/dvp-io.git

Harpy: saeyslab/harpy. Saeys Lab (2025). https://github.com/saeyslab/harpy.git

Lazyslide: Zheng, Y., Abila, E., Chrenková, E., Winkler, J. & Rendeiro, A. F. LazySlide: accessible and interoperable whole slide image analysis. 2025.05.28.656548 Preprint at https://doi.org/10.1101/2025.05.28.656548 (2025).

Snakemake Köster, J., Mölder, F., Jablonski, K. P., Letcher, B., Hall, M. B., Tomkins-Tinch, C. H., Sochat, V., Forster, J., Lee, S., Twardziok, S. O., Kanitz, A., Wilm, A., Holtgrewe, M., Rahmann, S., & Nahnsen, S. Sustainable data analysis with Snakemake. F1000Research, 10:33, 10, 33, 2021. https://doi.org/10.12688/f1000research.29032.2.

Spatialdata: Marconato, L. et al. SpatialData: an open and universal data framework for spatial omics. Nat Methods 1–5 (2024) doi:10.1038/s41592-024-02212-x.

Citation

DVP-Imaging-Pipeline [Computer software]. https://github.com/lucas-diedrich/dvp-imaging-pipeline.git