metacells 0.9.5

Single-cell RNA Sequencing Analysis

Metacells 0.9.5 - Single-cell RNA Sequencing Analysis

The metacells package implements the improved metacell algorithm [1] for single-cell RNA sequencing (scRNA-seq) data analysis within the scipy framework, and projection algorithm based on it [2]. The original metacell algorithm [3] was implemented in R. The python package contains various algorithmic improvements and is scalable for larger data sets (millions of cells).

Metacell Analysis

Naively, scRNA_seq data is a set of cell profiles, where for each one, for each gene, we get a count of the mRNA molecules that existed in the cell for that gene. This serves as an indicator of how “expressed” or “active” the gene is.

As in any real world technology, the raw data may suffer from technical artifacts (counting the molecules of two cells in one profile, counting the molecules from a ruptured cells, counting only the molecules from the cell nucleus, etc.). This requires pruning the raw data to exclude such artifacts.

The current technology scRNA-seq data is also very sparse (typically <<10% the RNA molecules are counted). This introduces large sampling variance on top of the original signal, which itself contains significant inherent biological noise.

Analyzing scRNA-seq data therefore requires processing the profiles in bulk. Classically, this has been done by directly clustering the cells using various methods.

In contrast, the metacell approach groups together profiles of the “same” biological state into groups of cells of the “same” biological state, with the minimal number of profiles needed for computing robust statistics (in particular, mean gene expression). Each such group is a single “metacell”.

By summing profiles of cells of the “same” state together, each metacell greatly reduces the sampling variance, and provides a more robust estimation of the transcription state. Note a metacell is not a cell type (multiple metacells may belong to the same “type”, or even have the “same” state, if the data sufficiently over-samples this state). Also, a metacell is not a parametric model of the cell state. It is merely a more robust description of some cell state.

The metacells should therefore be further analyzed as if they were cells, using additional methods to classify cell types, detect cell trajectories and/or lineage, build parametric models for cell behavior, etc. Using metacells as input for such analysis techniques should benefit both from the more robust, less noisy input; and also from the (~100-fold) reduction in the number of cells to analyze when dealing with large data (e.g. analyzing millions of individual cells).

A common use case is taking a new data set and using an existing atlas with annotations (in particular, “type” annotations) to provide initial annotations for the new data set. As of version 0.9 this capability is provided by this package.

Metacell projection provides a quantitative “projected” genes profile for each query metacell in the atlas, together with a “corrected” one for the same subset of genes shared between the query and the atlas. Actual correction is optional, to be used only if there are technological differences between the data sets, e.g. 10X v2 vs. 10X v3. This allows performing a quantitative comparison between the projected and corrected gene expression profiles for determining whether the query metacell is a novel state that does not exist in the atlas, or, if it does match an atlas state, analyze any differences it may still have. This serves both for quality control and for quantitative analysis of perturbed systems (e.g. knockouts or disease models) in comparison to a baseline atlas.

Terminology and Results Format

NOTE: Version 0.9 breaks compatibility with version 0.8 when it comes to some APIs and the names and semantics of the result annotations. See below for the description of updated results (and how they differ from version 0.8). The new format is meant to improve the usability of the system in downstream analysis pipelines. For convenience we also list here the results of the new projection pipeline added in version 0.9.*. Versions 0.9.1 and 0.9.2 contain some bug fixes. Version 0.9.3 allows specifying target UMIs for the metacells, in addition to the target size in cells, and adaptively tries to satisfy both. This should produce better-sized metacells “out of the box” compared to the 0.9.[0-2] versions. The latest published version, 0.9.4, contains minor bug fixes and updates for newer versions of dependency packages.

If you have existing metacell data that was computed using version 0.8 (the current published version you will get from using pip install metacells, you can use the provided conversion script script to migrate your data to the format described below, while preserving any additional annotations you may have created for your data (e.g. metacells type annotations). The script will not modify your existing data files, so you can examine the results and tweak them if necessary.

In an upcoming version we will migrate from using AnnData to using daf to represent the data (h5ad files will still be supported, either directly through an adapter or via a conversion process). This will again unavoidingly break API compatibility, but will provide many advantages over the restricted AnnData APIs.

We apologize for the inconvenience.

Metacells Computation

In theory, the only inputs required for metacell analysis are cell gene profiles with a UMIs count per gene per cell. In practice, a key part of the analysis is specifying lists of genes for special treatment. We use the following terminology for these lists:

excluded_gene, excluded_cell masks

Excluded genes (and/or cells) are totally ignored by the algorithm (e.g. mytochondrial genes, cells with too few UMIs).

Deciding on the “right” list of excluded genes (and cells) is crucial for creating high-quality metacells. We rely on the analyst to provide this list based on prior biological knowledge. To support this supervised task, we provide the excluded_genes and exclude_cells functions which implement “reasonable” strategies for detecting some (not all) of the genes and cells to exclude. For example, these will exclude any genes found by find_bursty_lonely_genes, (called find_noisy_lonely_genes in v0.8). Additional considerations might be to use relate_genes to (manually) exclude genes that are highly correlated with known-to-need-to-be-excluded genes, or exclude any cells that are marked as doublets, etc.

Currently the 1st step of the processing must be to create a “clean” data set which lacks the excluded genes and cells (e.g. using extract_clean_data). When we switch to daf we’ll just stay with the original data set and apply the exclusion masks to the rest of the algorithm.

lateral_gene mask

Lateral genes are forbidden from being selected for computing cells similarity (e.g., cell cycle genes). In version 0.8 these were called “forbidden” genes. Lateral genes are still counted towards the total UMIs count when computing gene expression levels for cells similarity. In addition, lateral genes are still used to compute deviant (outlier) cells. That is, each computed metacell should still have a consistent gene expression level even for lateral genes.

The motivation is that we don’t want the algorithm to even try to create metacells based on these genes. Since these genes may be very strong (again, cell cycle), they would overcome the cell-type genes we are interested in, resulting in for example an “M-state” metacell which combines cells from several (similar) cell types.

Deciding on the “right” list of lateral genes is crucial for creating high-quality metacells. We rely on the analyst to provide this list based on prior biological knowledge. To support this supervised task, we provide the relate_genes pipeline for identifying genes closely related to known lateral genes, so they can be added to the list.

noisy_gene mask

Noisy genes are given more freedom when computing deviant (outlier) cells. That is, we don’t expect the expression level of such genes in the cells in the same metacell to be as consistent as we do for regular (non-noisy) genes. Note this isn’t related to the question of whether the gene is lateral of not. That is, a gee maybe lateral, noisy, both, or neither.

The motivation is that some genes are inherently bursty and therefore cause many cells which are otherwise a good match for their metacell to be marked as deviant (outliers). An indication for this is by examining the deviant_fold matrix (see below).

Deciding on the “right” list of noisy genes is again crucial for creating high-quality metacells (and minimizing the fraction of outlier cells). Again we rely on the analyst here,

Having determined the inputs and possibly tweaking the hyper-parameters (a favorite one is the target_metacell_size, which by default is 160K UMIs; this may be reduced for small data sets and may be increased for larger data sets), one typically runs divide_and_conquer_pipeline to obtain the following:

metacell (index) vs. metacell_name (string) per cell

The result of computing metacells for a set of cells with the above assigns each cell a metacell index. We also give each metacell a name of the format M<index>.<checksum> where the checksum reflects the cells grouped into the metacell. This protects the analyst from mistakenly applying metadata assigned to metacells from an old computation to different newly computed metacells.

We provide functions (convey_obs_to_group, convey_group_to_obs) for conveying between per-cell and per-metacell annotations, which all currently use the metacell integer indices (this will change when we switch to daf). The metacell string names are safer to use, especially when slicing the data.

dissolve cells mask

Whether the cell was in a candidate matecall that was dissolved due to being too small (too few cells and/or total UMIs). This may aid quality control when there are a large number of outliers; lowering the target_metacell_size may help avoid this.

selected_gene mask

Whether each gene was ever selected to be used to compute the similarity between cells to compute the metacells. When using the divide-and-conquer algorithm, this mask is different for each pile (especially in the second phase when piles are homogeneous). This mask is the union of all the masks used in all the piles. It is useful for ensuring no should-be-lateral genes were selected as this would reduce the quality of the metacells. If such genes exist, add them to the lateral_gene mask and recompute the metacells.

Having computed the metacells, the next step is to run collect_metacells to create a new AnnData object for them (when using daf, they will be created in the same dataset for easier analysis), which will contain all the per-gene metadata, and also:

X per gene per metacell

Once the metacells have been computed (typically using divide_and_conquer_pipeline), we can collect the gene expression levels profile for each one. The main motivation for computing metacells is that they allow for a robust estimation of the gene expression level, and therefore we by default compute a matrix of gene fractions (which sum to 1) in each metacell, rather than providing a UMIs count for each. This simplifies the further analysis of the computed metacells (this is known as e_gc in the old R metacells package).

Note that the expression level of noisy genes is less reliable, as we do not guarantee the cells in each metacell have a consistent expression level for such genes. Our estimator therefore uses a normal weighted mean for most genes and a normalized geometric mean for the noisy gene. Since the sizes of the cells collected into the same metacell may vary, our estimator also ensures one large cell doesn’t dominate the results. That is, the computed fractions are not simply “sum of the gene UMIs in all cells divided by the sum of all gene UMIs in all cells”.

grouped per metacell

The number of cells grouped into each metacell.

total_umis per metacell, and per gene per metacell

We still provide the total UMIs count for each each gene for each cell in each metacell, and the total UMIs in each metacell. Note that the estimated fraction of each gene in the metacell is not its total UMIs divided by the metacell’s total UMIs; the actual estimator is more complex.

The total UMIs are important to ensure that analysis is meaningful. For example, comparing expression levels of lowly-expressed genes in two metacells will yield wildly inaccurate results unless a sufficient number of UMIs were used (the sum of UMIs of the gene in both compared metacells). The functions provided here for computing fold factors (log base 2 of the ratio) and related comparisons automatically ignore cases when this sum is below some threshold (40) by considering the effective fold factor to be 0 (that is, “no difference”).

metacells_level per cell or metacell

This is 0 for rare gene module metacells, 1 for metacells computed from the main piles in the 2nd divide-and-conquer phase and 2 for metacells computed for their outliers.

If using divide_and_conquer_pipeline, the following are also computed (but not by the simple compute_divide_and_conquer_metacells:

rare_gene_module_<N> mask (for N = 0, …)

A mask of the genes combined into each of the detected “rare gene modules”. This is done in (expensive) pre-processing before the full divide-and-conquer algorithm to increase the sensitivity of the method, by creating metacells computed only from cells that express each rare gene module.

rare_gene mask

A mask of all the genes in all the rare gene modules, for convenience.

rare_gene_module per cell or metacell

The index of the rare gene module each cell or metacell expresses (or negative for the common case it expresses none of them).

rare_cell, rare_metacell masks

A mask of all the cells or metacells expressing any of the rare gene modules, for convenience.

In theory one is free to go use the metacells for further analysis, but it is prudent to perform quality control first. One obvious measure is the number of outlier cells (with a negative metacell index and a metacell name of Outliers). In addition, one should compute and look at the following (an easy way to compute all of them at once is to call compute_for_mcview, this will change in the future):

most_similar, most_similar_name per cell (computed by compute_outliers_most_similar)

For each outlier cell (whose metacell index is -1 and metacell name is Outliers), the index and name of the metacell which is the “most similar” to the cell (has highest correlation).

deviant_fold per gene per cell (computed by compute_deviant_folds)

For each cell, for each gene, the deviant_fold holds the fold factor (log base 2) between the expression level of the gene in the cell and the metacell it belongs to (or the most similar metacell for outlier cells). This uses the same (strong) normalization factor we use when computing deviant (outlier) cells, so for outliers, you should see some (non-excluded, non-noisy) genes with a fold factor above 3 (8x), or some (non-excluded, noisy) genes with a fold factor above 5 (32x), which justify why we haven’t merged that cell into a metacell; for cells grouped into metacells, you shouldn’t see (many) such genes. If there is a large number of outlier cells and a few non-noisy genes have a high fold factor for many of them, you should consider marking these genes as noisy and recomputing the metacells. If they are already marked as noisy, you may want to completely exclude them.

inner_fold per gene per metacell (computed by compute_inner_folds)

For each metacell, for each gene, the inner_fold is the strongest (highest absolute value) deviant_fold of any of the cells contained in the metacell. Both this and the inner_stdev_log below can be used for quality control over the consistency of the gene expression in the metacell.

significant_inner_folds_count per gene

For each gene, the number of metacells in which there’s at least one cell with a high deviant_fold (that is, where the inner_fold is high). This helps in identifying troublesome genes, which can be then marked as noisy, lateral or even excluded, depending on their biological significance.

inner_stdev_log per gene per metacell (computed by compute_inner_stdev_logs)

For each metacell, for each gene, the standard deviation of the log (base 2) of the fraction of the gene across the cells of the metacell. Ideally, the standard deviation should be ~1/3rd of the deviants_min_gene_fold_factor (which is 3 by default), indicating that (all)most cells are within that maximal fold factor. In practice we may see higher values - the lower, the better. Both this and the inner_fold above can be used for quality control over the consistency of the gene expression in the metacell.

marker_gene mask (computed by find_metacells_marker_genes)

Given the computed metacells, we can identify genes that have a sufficient number of effective UMIs (in some metacells) and also have a wide range of expressions (between different metacells). These genes serve as markers for identifying the “type” of the metacell (or, more generally, the “gene programs” that are active in each metacell).

Typically analysis groups the marker genes into “gene modules” (or, more generally, “gene programs”), and then use the notion of “type X expresses the gene module/programs Y, Z, …”. As of version 0.9, collecting such gene modules (or programs) is left to the analyst with little or no direct support in this package, other than providing the rare gene modules (which by definition would apply only to a small subset of the metacells).

x, y per metacell (computed by compute_umap_by_markers)

A common and generally effective way to visualize the computed metacells is to project them to a 2D view. Currently we do this by giving UMAP a distance metric between metacells based on a logistic function based on the expression levels of the marker genes. In version 0.8 this was based on picking (some of) the selected genes.

This view is good for quality control. If it forces “unrelated” cell types together, this might mean that more genes should be made lateral, or noisy, or even excluded; or maybe the data contains a metacell of doublets; or metacells mixing cells from different types, if too many genes were marked as lateral or noisy, or excluded. It takes a surprising small number of such doublet/mixture metacells to mess up the UMAP projection.

Also, one shouldn’t read too much from the 2D layout, as by definition it can’t express the “true” structure of the data. Looking at specific gene-gene plots gives much more robust insight into the actual differences between the metacell types, identify doublets, etc.

obs_outgoing_weights per metacell per metacell (also computed by compute_umap_by_markers)

The (sparse) matrix of weights of the graph used to generate the x and y 2D projection. This graph is very sparse, that is, has a very low degree for the nodes. It is meant to be used only in conjunction with the 2D coordinates for visualization, and should not be used by any downstream analysis to determine which metacells are “near” each other for any other purpose.

Metacells Projection

For the use case of projecting metacells we use the following terminology:

atlas

A set of metacells with associated metadata, most importantly a type annotation per metacell. In addition, the atlas may provide an essential_gene_of_<type> mask for each type. For a query metacell to successfully project to a given type will require that the query’s expression of the type’s essential genes matches the atlas. We also use the metadata listed above (specifically, lateral_gene, noisy_gene and marker_gene).

query

A set of metacells with minimal associated metadata, specifically without a type. This may optionally contain its own lateral_gene, noisy_gene and/or even marker_gene annotations.

ignored_gene mask, ignored_gene_of_<type> mask

A set of genes to not even try to match between the query and the atlas. In general the projection matches only a subset of the genes (that are common to the atlas and the query). However, the analyst has the option to force additional genes to be ignored, either in general or only when projecting metacells of a specific type. Manually ignoring specific genes which are known not to match (e.g., due to the query being some experiment, e.g. a knockout or a disease model) can improve the quality of the projection for the genes which do match.

Given these two input data sets, the projection_pipeline computes the following (inside the query AnnData object):

atlas_gene mask

A mask of the query genes that also exist in the atlas. We match genes by their name; if projecting query data from a different technology, we expect the caller to modify the query gene names to match the atlas before projecting it.

atlas_lateral_gene, atlas_noisy_gene, atlas_marker_gene, essential_gene_of_<type> masks

These masks are copied from the atlas to the query (restricting them to the common atlas_gene subset).

projected_noisy_gene

The mask of the genes that were considered “noisy” when computing the projection. By default this is the union of the noisy atlas and query genes.

corrected_fraction per gene per query metacell

For each atlas_gene, its fraction in each query metacell, out of only the atlas genes. This may be further corrected (see below) if projecting between different scRNA-seq technologies (e.g. 10X v2 and 10X v3). For non-atlas_gene this is 0.

projected_fraction per gene per query metacell

For each atlas_gene, its fraction in its projection on the atlas. This projection is computed as a weighted average of some atlas metacells (see below), which are all sufficiently close to each other (in terms of gene expression), so averaging them is reasonable to capture the fact the query metacell may be along some position on some gradient that isn’t an exact match for any specific atlas metacell. For non-atlas_gene this is 0.

total_atlas_umis per query metacell

The total UMIs of the atlas_gene in each query metacell. This is used in the analysis as described for total_umis above, that is, to ensure comparing expression levels will ignore cases where the total number of UMIs of both compared gene profiles is too low to make a reliable determination. In such cases we take the fold factor to be 0.

weights per query metacell per atlas metacsll

The weights used to compute the projected_fractions. Due to AnnData limitations this is returned as a separate object, but in daf we should be able to store this directly into the query object.

In theory, this would be enough for looking at the query metacells and comparing them to the atlas, and to project metadata from the atlas to the query (e.g., the metacell type) using convey_atlas_to_query. In practice, there is significant amount of quality control one needs to apply before accepting these results, which we compute as follows:

correction_factor per gene

If projecting a query on an atlas with different technologies (e.g., 10X v3 to 10X v2), an automatically computed factor we multiplied the query gene fractions by to compensate for the systematic difference between the technologies (1.0 for uncorrected genes and 0.0 for non-atlas_gene).

projected_type per query metacell

For each query metacell, the best atlas type we can assign to it based on its projection. Note this does not indicate that the query metacell is “truly” of this type; to make this determination one needs to look at the quality control data below.

projected_secondary_type per query metacell

In some cases, a query metacell may fail to project well to a single region of the atlas, but does project well to a combination of two distinct atlas regions. This may be due to the query metacell containing doublets, of a mixture of cells which match different atlas regions (e.g. due to sparsity of data in the query data set). Either way, if this happens, we place here the type that best describes the secondary region the query metacell was projected to; otherwise this would be the empty string. Note that the weights matrix above does not distinguish between the regions.

fitted_gene_of_<type> mask

For each type, the genes that were projected well from the query to the atlas for most cells of that type; any atlas_gene outside this mask failed to project well from the query to the atlas for most metacells of this type. For non-atlas_gene this is set to False.

Whether failing to project well some of the atlas_gene for most metacells of some projected_type indicates that they aren’t “truly” of that type is a decision which only the analyst can make based, on prior biological knowledge of the relevant genes.

fitted mask per gene per query metacell

For each atlas_gene for each query metacell, whether the gene was expected to be projected well, based on the query metacell projected_type (and the projected_secondary_type, if any). For non-atlas_gene this is set to False. This does not guarantee the gene was actually projected well.

misfit mask per gene per query metacell

For each atlas_gene for each query metacell, whether the corrected_fraction of the gene was significantly different from the projected_fractions (that is, whether the gene was not projected well for this metacell). For non-atlas_gene this is set to False, to make it easier to identify problematic genes.

This is expected to be rare for fitted genes and common for the rest of the atlas_gene. If too many fitted genes are also misfit, then one should be suspicious whether the query metacell is “truly” of the projected_type.

essential mask per gene per query metacell

Which of the atlas_gene were also listed in the essential_gene_of_<type> for the projected_type (and also the projected_secondary_type, if any) of each query metacell.

If an essential gene is also a misfit gene, then one should be very suspicious whether the query metacell is “truly” of the projected_type.

projected_correlation per query metacell

The correlation between between the corrected_fraction and the projected_fraction for only the fitted genes expression levels of each query metacell. This serves as a very rough estimator for the quality of the projection for this query metacell (e.g. can be used to compute R^2 values).

In general we expect high correlation (more than 0.9 in most metacells) since we restricted the fitted genes mask only to genes we projected well.

projected_correlation per gene

For every gene (not only fitted genes), the correlation between the corrected_fraction and projected_fraction across all the query metacells. In general we expect high correlation for fitted genes and low correlation for the rest.

projected_fold per gene per query metacell

The fold factor between the corrected_fraction and the projected_fraction (0 for non-atlas_gene). If the absolute value of this is high (3 for 8x ratio) then the gene was not projected well for this metacell. This will be 0 for non-atlas_gene, or if the total number of UMIs is too low.

It is expected this would have low values for most fitted genes and high values for the rest of the atlas_gene, but specific values will vary from one query metacell to another. This allows the analyst to make fine-grained determination about the quality of the projection, and/or identify quantitative differences between the query and the atlas (e.g., when studying perturbed systems such as knockouts or disease models).

similar mask per query metacell

A conservative determination of whether the query metacell is “similar” to its projection on the atlas. This is based on whether the number of misfit for the query metacell is low enough (by default, up to 3 genes), and also that at least 75% of the essential genes of the query metacell were not misfit genes. Note that this explicitly allows for a projected_secondary_type, that is, a metacell of doublets will be “similar” to the atlas, but a metacell of a novel state missing from the atlas will be “dissimilar”.

The final determination of whether to accept the projection is, as always, up to the analyst, based on prior biological knowledge, the context of the collection of the query (and atlas) data sets, etc. The analyst need not (indeed, should not) blindly accept the similar determination without examining the rest of the quality control data listed above.

Installation

In short: pip install metacells. Note that metacells requires many “heavy” dependencies, most notably numpy, pandas, scipy, scanpy, which pip should automatically install for you. If you are running inside a conda environment, you might prefer to use it to first install these dependencies, instead of having pip install them from PyPI.

Note that metacells only runs natively on Linux and MacOS. To run it on a Windows computer, you must activate Windows Subsystem for Linux and install metacells within it.

The metacells package contains extensions written in C++. The metacells distribution provides pre-compiled Python wheels for both Linux and MacOS, so installing it using pip should not require a C++ compilation step.

Note that for X86 CPUs, these pre-compiled wheels were built to use AVX2 (Haswell/Excavator CPUs or newer), and will not work on older CPUs which are limited to SSE. Also, these wheels will not make use of any newer instructions (such as AVX512), even if available. While these wheels may not the perfect match for the machine you are running on, they are expected to work well for most machines.

To see the native capabilities of your machine, you can grep flags /proc/cpuinfo | head -1 which will give you a long list of supported CPU features in an arbitrary order, which may include sse, avx2, avx512, etc. You can therefore simply grep avx2 /proc/cpuinfo | head -1 to test whether AVX2 is/not supported by your machine.

You can avoid installing the pre-compiled wheel by running pip install metacells --no-binary :all:. This will force pip to compile the C++ extensions locally on your machine, optimizing for its native capabilities, whatever these may be. This will take much longer but may give you somewhat faster results (note: the results will not be exactly the same as when running the precompiled wheel due to differences in floating-point rounding). Also, this requires you to have a C++ compiler which supports C++14 installed (either g++ or clang). Installing a C++ compiler depends on your specific system (using conda may make this less painful).

Vignettes

The latest vignettes can be found here.

References

Please cite the references appropriately in case they are used:

[1]

Ben-Kiki, O., Bercovich, A., Lifshitz, A. et al. Metacell-2: a divide-and-conquer metacell algorithm for scalable scRNA-seq analysis. Genome Biol 23, 100 (2022). https://doi.org/10.1186/s13059-022-02667-1

[2]

Ben-Kiki, O., Bercovich, A., Lifshitz, A. et al. MCProj: metacell projection for interpretable and quantitative use of transcriptional atlases. Genome Biol 24, 220 (2023). https://doi.org/10.1186/s13059-023-03069-7

[3]

Baran, Y., Bercovich, A., Sebe-Pedros, A. et al. MetaCell: analysis of single-cell RNA-seq data using K-nn graph partitions. Genome Biol 20, 206 (2019). 10.1186/s13059-019-1812-2

License (MIT)

Copyright © 2020-2023 Weizmann Institute of Science

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

History

0.5

• First published version.

0.6

• More robust graph partition.

• Allow forcing feature genes.

• Rename “project” to “convey” to prepare for addition of atlas projection functionality.

0.7.0

• Switch to new project template.

• Fix some edge cases in the pipeline.

• Switch to using psutil for detecting system resources.

• Fix binary wheel issues.

• Give up on using travis-ci.

0.8.0

• Add inner fold factor computation for metacells quality control.

• Add deviant fold factor computation for metacells quality control.

• Add projection of query data onto an atlas.

• Self-adjusting pile sizes.

• Add convenience function for computing data for MCView.

• Better control over filtering using absolute fold factors.

• Fix edge case in computing noisy lonely genes.

• Additional outliers certificates.

• Stricter deviants detection policy

0.9.0

• Improved and published projection algorithm.

• Restrict number of rare gene candidates.

• Tighter control over metacells size and internal quality.

• Improved divide-and-conquer strategy.

• Base deviants (outliers) on gaps between cells.

• Terminology changes (see the README for details).

• Projection!

0.9.1

• Fix build for python 3.11.

• More robust gene selection, KNN graph creation, and metacells partition.

• More thorough binary wheels.

0.9.2

• Fix numpy compatibility issue.

• Fix K of UMAP skeleton KNN graph (broken in 0.9.1).

0.9.3

• Allow specifying both target UMIs and target size (in cells) for the metacells, and adaptively try to satisfy both. This should produce better-sized metacells “out of the box” compared to 0.9.[0-2].

0.9.4

• Fix minor bug in regularization of metacell fractions.

• Fix issue with canonical sparse matrices after downsampling (probably due to scipy.sparse updates?)

• Fix using deprecated AnnData APIs.

0.9.5

• Improve recovery from unstable convex solvers.

• Fix an edge case in computing deviant cells using the gaps policy.

• Turns out the threadpoolctl is not thread-safe (the irony!), causing deadlocks. Work around this.