This release is a pre-release and may not be stable for production use.
Scarf
Single Cell Analysis on Remote Filesystems
Scarf is a Python framework for analysing single-cell RNA, ATAC, protein, and multi-omic data, from a few thousand cells to tens of millions.
| Problem | How Scarf solves it | What you get |
|---|---|---|
| Your dataset is larger than RAM | Out-of-core algorithms, and neighbour search streams from cell-major and gene-major layouts, inside a memory budget you set | No subsampling, so rare populations survive, benchmarked to 10M cells |
| The data is stored remotely and requires downloading | Fetches only the chunks an operation touches, and writes results to a store you own | Start analysing immediately, with one authoritative copy |
| A single parameter change costs hours of computation | Each step is fingerprinted by its settings and inputs, so reuse is by content, not by layer name | Only what changed recomputes, and the old version stays for comparison |
| Sub-population analysis leaves scattered copies that nobody can trace back | Subsets are masks in one file, and every result carries the cells and parameters behind it | A year later, a result still explains itself |
Install
Python 3.12+.
uv venv --python 3.12
uv pip install --python .venv "scarf[extra]"
Detailed installation instructions here
Quick start
import scarf
ds = scarf.DataStore(
"s3://bucket/10M_cells.zarr", # also gs://, hf://, or a local path
)
run = ds.pipeline.run() # durable QC → graph → UMAP → clustering → marker run
ds.plots.embedding(
run=run,
layout="umap",
color_by="clusters",
)
Read the scRNA-seq tutorial for a granular workflow, or remote stores for cloud setups.
Documentation
Read workflow vignettes and API references on Read The Docs 📖
AI-assisted and autonomous workflows should start with Analysis with AI agents.
Scarf's capabilities
| Area | Methods |
|---|---|
| Modalities | scRNA-seq, scATAC-seq, CITE-seq, matched multi-omics |
| Core workflow | Quality control, feature selection, normalization, PCA and LSI, KNN graph, UMAP, densMAP, t-SNE, Leiden, Paris, marker search |
| Integration | Harmony, partial PCA, shared and weighted nearest neighbours, integration metrics |
| Mapping | Symphony-style reference mapping, label transfer, projection diagnostics |
| Trajectory | Population Balance Analysis pseudotime, expression dynamics and modules, multi-sink fate probabilities |
| Also included | Cell-cycle scoring, gene-set activity, graph-diffusion imputation, doublet scores, HTO demultiplexing, TopACeDo downsampling, pseudobulk export |
Citation
Dhapola et al. Scarf enables a highly memory-efficient analysis of large-scale single-cell genomics data. Nature Communications 13, 4616 (2022).
Support
Scarf is open source software released under the BSD 3-Clause License and maintained by Nygen.
1 maintainer
Metadata
1 maintainer
Release files for scarf 1.0.0rc18
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| scarf-1.0.0rc18.tar.gz | 1.6 MB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| scarf-1.0.0rc18-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 3.1 MB
Release files / scarf-1.0.0rc18.tar.gz
| Download URL | scarf-1.0.0rc18.tar.gz |
|---|---|
| Size | 1.6 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
c6debcfe59071546fca4ff9372ba77643508e8841b15111bebc1520c5a2a8ce4
|
|
BLAKE2b-256 checksum How to use checksums |
fc7e2c01f504c104fc70e2f91f30ccb85f49ff5ec81bb9128d00f0d2f9aa1069
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 2, 2026.
Transparency logRelease files / scarf-1.0.0rc18-py3-none-any.whl
| Download URL | scarf-1.0.0rc18-py3-none-any.whl |
|---|---|
| Size | 1.5 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
9ac942cab81c78bca2c1ca7411d8daac2f8be954dd2e503f00c59ed3fba24e68
|
|
BLAKE2b-256 checksum How to use checksums |
72a6c630fa200f65115b2e8585c3095ef2b763cdfd18423639326a94e1c5aae8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 2, 2026.
Transparency log
