SHERLOCK: structured representation learning and causal inference of downstream perturbation effects

SHERLOCK (Single-cell Hierarchical Embedding of peRturbations and Learning Of Causal Knowledge) is an interpretable deep generative framework for single-cell perturbation data. It models each genetic or chemical perturbation as a structured intervention on a latent baseline cell state, with perturbation effects that are correlated across perturbations and sparse across latent factors. From the fitted model you can

• group perturbations by the transcriptional responses they share,
• estimate downstream effects counterfactually, by applying a perturbation to control cells' inferred baseline states and decoding the result,
• measure how responses depend on a condition such as drug exposure or T-cell co-culture, and
• compose single perturbations to predict combinations, including unseen ones, and classify genetic interactions.

📖 Documentation: https://azizilabsherlock.readthedocs.io

Installation

SHERLOCK requires Python ≥ 3.10 and is published on PyPI as azizilab-sherlock; the package is imported as sherlock.

pip install azizilab-sherlock

Optional extras:

pip install "azizilab-sherlock[tutorials]"

To install from source, or with conda:

git clone https://github.com/azizilab/SHERLOCK.git
cd SHERLOCK
pip install -e ".[tutorials]"          # or: conda env create -f environment.yml

Install PyTorch first if you need a specific CUDA build (pytorch.org). See the installation guide for details.

Quickstart

import scanpy as sc
import torch
import sherlock as slk

# Raw counts in .X; one perturbation label per cell, controls labelled e.g. "non-targeting"
adata = sc.read_h5ad("my_screen.h5ad")
slk.pp.slk_prepare_data(adata, pert_key="guide_target", ntc_label="non-targeting", inplace=True)

# Observed average treatment effects, used for validation during training
slk.pp.treat_effect(adata, label_col="pert", control_label="NTC", method="perturbseq", inplace=True)

# Train and evaluate
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
out = slk.tl.run(adata, model="vae", latent_dim=16, device=device)
model = out["model"]
slk.tl.eval(model, adata, device=device)       # writes adata.obsm["z"] and adata.uns["results"]

# Perturbation groups and counterfactual downstream effects
slk.pl.plot_corr(adata)
cf = model.predict_counterfactual_effects(adata, observed_effect=adata.uns["treat_effect"])
slk.tl.ev_sig(adata)
slk.pl.plot_ev_sig(adata)

slk.tl.save_results(out, "sherlock_model.pth")

For screens with conditions, also pass treatment_key and untreated_label to slk_prepare_data and train with use_conditions=True. For combinations (A+B labels), train with combinatorial=True, synergy=True. See the API reference.

Tutorials

The tutorial notebooks are in docs/tutorials/.

Datasets

The processed Replogle and Norman datasets are downloaded on first use:

adata = slk.datasets.replogle()   # ~570 MB
adata = slk.datasets.norman()     # ~1.7 GB

Files are stored in $SHERLOCK_DATA_DIR (default ~/.cache/sherlock); pass path= to read a local copy instead. The processed versions of the other datasets analysed in the manuscript (Perturb-RAEFISH, EGFR inhibitor screen, glioblastoma kinome and kinase-inhibitor screens) are available from the first authors on request.

Repository structure

The repository root is the sherlock package:

├── __init__.py, _configs.py
├── tools/           # models, training, evaluation, clustering, interaction scoring (slk.tl)
├── preprocessing/   # label preparation and observed treatment effects (slk.pp)
├── plotting/        # plots (slk.pl)
├── datasets/        # dataset loaders (slk.datasets); downloaded data is not tracked
├── notebooks/       # analysis notebooks for every dataset in the manuscript
│   ├── replogle_analysis.ipynb, benchmarking.ipynb
│   ├── raefish/     # spatial CRISPR screen
│   ├── gbm/         # glioblastoma: EGFR inhibitors, kinome CRISPRi/a, 11 drugs ± T cells
│   └── norman/      # combinatorial CRISPRa screen
├── docs/            # documentation source: guides, API reference, tutorials
├── tests/           # unit tests
├── pyproject.toml   # packaging (pip install)
└── environment.yml  # conda environment

The notebooks import the installed package, so run pip install -e . in the repository root before opening them. They read their inputs from datasets/. A version organized by figure, with instructions for obtaining the data and saved models, is in azizilab/SHERLOCK_Reproducibility.

Citation

If you use SHERLOCK, please cite our preprint: https://www.biorxiv.org/content/10.64898/2026.09.25.754573v1

SHERLOCK: Structured representation learning and causal inference of downstream perturbation effects

@article{zhang2026sherlock,
  title   = {SHERLOCK: Structured representation learning and causal inference of downstream perturbation effects},
  author  = {Zhang, Mingxuan and Myers, Joshua D. and Shi, Lingting and Giglio, Ross M. and
             Chatterjee, Sharanya and McFaline-Figueroa, Jos{\'e} L. and Azizi, Elham},
  journal = {bioRxiv},
  year    = {2026},
  doi     = {10.64898/2026.09.25.754573}
}

License

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.