This release is a pre-release and may not be stable for production use.
Scarf
Single Cell Analysis on Remote Filesystems
[!IMPORTANT] Scarf 1.0 is coming soon. Install the
release candidate with
uv pip install --prerelease allow "scarf[extra]". The current stable release on PyPI is0.32.3.
Scarf is a Python framework for analysing single-cell RNA, ATAC, protein, and multi-omic data, from a few thousand cells to tens of millions.
| Problem | How Scarf solves it | What you get |
|---|---|---|
| Your dataset is larger than RAM | Out-of-core algorithms, and neighbour search streams from cell-major and gene-major layouts, inside a memory budget you set | No subsampling, so rare populations survive, benchmarked to 10M cells |
| The data is stored remotely and requires downloading | Fetches only the chunks an operation touches, and writes results to a store you own | Start analysing immediately, with one authoritative copy |
| A single parameter change costs hours of computation | Each step is fingerprinted by its settings and inputs, so reuse is by content, not by layer name | Only what changed recomputes, and the old version stays for comparison |
| Sub-population analysis leaves scattered copies that nobody can trace back | Subsets are masks in one file, and every result carries the cells and parameters behind it | A year later, a result still explains itself |
Install
Python 3.12+.
uv venv --python 3.12
uv pip install --python .venv "scarf[extra]"
Detailed installation instructions here
Quick start
import scarf
ds = scarf.DataStore(
"s3://bucket/10M_cells.zarr", # also gs://, hf://, or a local path
)
ds.pipeline.run() # convenience: QC → HVGs → PCA → graph → UMAP → clustering → markers
ds.plots.embedding(
layout_key="RNA_UMAP",
color_by="RNA_clusters",
)
Read the scRNA-seq tutorial for a granular workflow, or remote stores for cloud setups.
Documentation
Read workflow vignettes and API references on Read The Docs 📖
AI-assisted and autonomous workflows should start with Analysis with AI agents.
Scarf's capabilities
| Area | Methods |
|---|---|
| Modalities | scRNA-seq, scATAC-seq, CITE-seq, matched multi-omics |
| Core workflow | Quality control, feature selection, normalization, PCA and LSI, KNN graph, UMAP, densMAP, t-SNE, Leiden, Paris, marker search |
| Integration | Harmony, partial PCA, shared and weighted nearest neighbours, integration metrics |
| Mapping | Symphony-style reference mapping, label transfer, projection diagnostics |
| Trajectory | Population Balance Analysis pseudotime, expression dynamics and modules, multi-sink fate probabilities |
| Also included | Cell-cycle scoring, gene-set activity, graph-diffusion imputation, doublet scores, HTO demultiplexing, TopACeDo downsampling, pseudobulk export |
Citation
Dhapola et al. Scarf enables a highly memory-efficient analysis of large-scale single-cell genomics data. Nature Communications 13, 4616 (2022).
Support
Scarf is open source software released under the BSD 3-Clause License and maintained by Nygen.
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file scarf-1.0.0rc4.tar.gz.
File metadata
- Download URL: scarf-1.0.0rc4.tar.gz
- Upload date:
- Size: 1.0 MB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
363746d39bceddb300a22040b4144c2c73fd2a0d39cebb18b04b90ed46a0d714
|
|
| MD5 |
8dcaf2b3d6ebd98081bcab1c1871581e
|
|
| BLAKE2b-256 |
209b61cf42dd6d1ec71df7f0dfe3c8582c010b00d49602de03b5dfc7dec740ef
|
Provenance
The following attestation bundles were made for scarf-1.0.0rc4.tar.gz:
Publisher:
publish.yml on NygenAnalytics/scarf
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
scarf-1.0.0rc4.tar.gz -
Subject digest:
363746d39bceddb300a22040b4144c2c73fd2a0d39cebb18b04b90ed46a0d714 - Sigstore transparency entry: 2368760005
- Sigstore integration time:
-
Permalink:
NygenAnalytics/scarf@5e0461ebeceecd27bc576643ef530e2a7373bae5 -
Branch / Tag:
refs/tags/1.0.0rc4 - Owner: https://github.com/NygenAnalytics
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish.yml@5e0461ebeceecd27bc576643ef530e2a7373bae5 -
Trigger Event:
release
-
Statement type:
File details
Details for the file scarf-1.0.0rc4-py3-none-any.whl.
File metadata
- Download URL: scarf-1.0.0rc4-py3-none-any.whl
- Upload date:
- Size: 870.0 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
f09f4091fe02603aba56907c3075cf5011a1d6adb555b1c80b4f035709958c4a
|
|
| MD5 |
c09191d3a227c0e0dc6ba18aa5402282
|
|
| BLAKE2b-256 |
99a604a7103a594aa08bbc6e65473473be0ede89c0312751fc64bbe89abb0ef0
|
Provenance
The following attestation bundles were made for scarf-1.0.0rc4-py3-none-any.whl:
Publisher:
publish.yml on NygenAnalytics/scarf
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
scarf-1.0.0rc4-py3-none-any.whl -
Subject digest:
f09f4091fe02603aba56907c3075cf5011a1d6adb555b1c80b4f035709958c4a - Sigstore transparency entry: 2368760064
- Sigstore integration time:
-
Permalink:
NygenAnalytics/scarf@5e0461ebeceecd27bc576643ef530e2a7373bae5 -
Branch / Tag:
refs/tags/1.0.0rc4 - Owner: https://github.com/NygenAnalytics
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish.yml@5e0461ebeceecd27bc576643ef530e2a7373bae5 -
Trigger Event:
release
-
Statement type:
