Skip to main content

crispyx

License: Modified MIT Python 3.10+ PyPI PyPI Downloads Tests

Motivation

Genome-wide CRISPR screens routinely produce datasets with hundreds of thousands of cells and tens of thousands of genes. Standard single-cell analysis toolkits (Scanpy, Pertpy) load the entire count matrix into memory, requiring large RAM allocations and often making routine workflows impractical on laptops or shared compute nodes.

crispyx solves this by streaming data directly from on-disk AnnData (.h5ad) files. Quality control, normalisation, pseudo-bulk aggregation, and differential expression all operate without materialising the full matrix.

Features

  • Streaming QC & preprocessing – Filter cells, perturbations, and genes; normalise and log-transform; CSC-aware streaming with format_mismatch_policy; all without loading the full matrix into memory
  • Subsampling & downsampling – Stratified or cluster-sampled cell subsampling (cx.pp.subsample, exact count or proportion per stratum, drop or keep small groups) and dependency-free per-cell count thinning (cx.pp.downsample_counts, the streaming equivalent of scanpy.pp.downsample_counts) for aligning dataset scale and sequencing depth before comparing screens
  • Pseudo-bulk aggregation – Absolute profiles over multiple grouping columns (for example, perturbation × batch), strict count sums or mean log1p expression, optional deterministic bootstrap sampling, and explicit within-batch effect calculation
  • Differential expression – t-test, Wilcoxon rank-sum (including batch-stratified / van Elteren test via batch_column), and negative binomial GLM with apeGLM LFC shrinkage; multi-core support and adaptive memory management; per-condition low-expression filtering to exclude genes that are near-zero in both groups
  • Dimension reduction – Memory-efficient PCA and KNN graph construction on backed data
  • Scanpy-compatible API & plotting – Familiar cx.pp, cx.pb, cx.tl, and cx.pl namespaces; Scanpy-style rank genes plots, volcano, MA, PCA, UMAP, QC summaries, and overlap heatmaps
  • Data preparation utilities – Edit backed metadata without loading X; standardise gene names; normalise perturbation labels; auto-detect metadata columns
  • HPC-ready – Resume/checkpoint for long-running jobs; configurable memory_limit_gb
  • Disk-aware – Estimates and warns about scratch-disk usage before large writes or CSC/CSR conversions, and cx.estimate_disk_usage(...) answers "how much disk will this need?" up front; the memory savings above assume the machine has enough free disk for streaming intermediates and output files

Quick Start

import crispyx as cx

# Open dataset without loading into memory
adata = cx.read_h5ad_ondisk("data/demo_benchmark.h5ad")

# Quality control with adaptive thresholds
adata = cx.pp.qc_summary(
    adata,
    perturbation_column="perturbation",
    min_genes=5,
    min_cells_per_perturbation=5,
)

# Differential expression
adata = cx.tl.rank_genes_groups(
    adata,
    perturbation_column="perturbation",
    method="wilcoxon",  # or "t-test", "nb_glm"
)

# Access results
print(adata.uns["rank_genes_groups"])
de_results = adata.uns["rank_genes_groups"].load()

For the full workflow (normalisation, PCA, pseudo-bulk, NB-GLM, LFC shrinkage, plotting, data preparation utilities), see the Usage Guide and the tutorial notebook.

Performance

crispyx consistently outperforms Scanpy, Pertpy/PyDESeq2, and edgeR in both speed and memory across a range of CRISPR screen dataset sizes, with results matching Scanpy to Pearson r > 0.999:

Benchmark results across 12 CRISPR screens: (a) dataset sizes, (b) completion status by method, (c) concordance with Scanpy, (d) runtime scaling, (e) peak memory scaling

Installation

pip install crispyx

For development (editable install with all extras):

git clone https://github.com/jinhongdu-lab/crispyx.git
cd crispyx
pip install -e ".[test,benchmark,docs]"

crispyx supports Python 3.10–3.12 and is compatible with recent releases of the scientific stack, including anndata >= 0.13 and pandas >= 3.0 (where string metadata is stored on disk using the nullable-string encoding).

Testing

pytest

Documentation

sphinx-build docs docs/_build

Acknowledgements

crispyx builds on the foundational work of Scanpy (Wolf et al., 2018), Pertpy, PyDESeq2 (Muzellec et al., 2023), and AnnData (Virshup et al., 2024). We gratefully acknowledge these projects for establishing the single-cell analysis ecosystem in Python; crispyx extends their APIs and algorithmic designs to enable memory-efficient, streaming computation for large-scale CRISPR screen datasets.

Contributing

Suggestions, bug reports, and contributions are welcome! Please open an issue or submit a pull request.

License

crispyx is released under a Modified MIT License. If you use crispyx in research, please cite it — see CITATION.cff.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

crispyx-0.1.1.tar.gz (337.8 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

crispyx-0.1.1-py3-none-any.whl (260.4 kB view details)

Uploaded Python 3

File details

Details for the file crispyx-0.1.1.tar.gz.

File metadata

  • Download URL: crispyx-0.1.1.tar.gz
  • Upload date:
  • Size: 337.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for crispyx-0.1.1.tar.gz
Algorithm Hash digest
SHA256 ab04d48943846104f0f00cbe5aa7eba7ccc60daea8a8441b669279612afd3ba9
MD5 d26a029ce5dc5e32ed3f603bd65a4b38
BLAKE2b-256 38e05a38f69104cea6f644b6984e9a7f2b399c35d0f8ed06c380a59441adb6a3

See more details on using hashes here.

Provenance

The following attestation bundles were made for crispyx-0.1.1.tar.gz:

Publisher: publish.yml on JinHongDu-Lab/crispyx

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file crispyx-0.1.1-py3-none-any.whl.

File metadata

  • Download URL: crispyx-0.1.1-py3-none-any.whl
  • Upload date:
  • Size: 260.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for crispyx-0.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 07cec78bc8c1acdbae1489005fa2ba3e3f68ff19e55a54762b5b4f483651fb8e
MD5 d6c3159a1eda5fbc358a2b8c1b81c507
BLAKE2b-256 6478f3056c1af19b29ce9d1430860c910bc6f03468a4aa074f532fa10dac51bc

See more details on using hashes here.

Provenance

The following attestation bundles were made for crispyx-0.1.1-py3-none-any.whl:

Publisher: publish.yml on JinHongDu-Lab/crispyx

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.1.2

2 files

This release

0.1.1 This release

2 files

0.1.0

2 files

0.0.10

2 files

0.0.9

2 files

0.0.8

2 files

0.0.7

2 files

0.0.6

2 files

0.0.5

2 files

0.0.4

2 files

0.0.3

2 files

0.0.2

2 files

0.0.1

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page