Skip to main content

Python library to handle stacks of sparse COO arrays efficiently.

Project description

GitHub PyPI Conda GitHub Workflow Status fair-software.eu

sparsestack logo

Memory efficient stack of multiple 2D sparse arrays.

sparsestack-overview-figure

Installation

Requirements

Python 3.8 or higher

Pip Install

Simply install using pip: pip install sparsestack

First code example

import numpy as np
from sparsestack import StackedSparseArray

# Create some fake data
scores1 = np.random.random((12, 10))
scores1[scores1 < 0.9] = 0  # make "sparse"
scores2 = np.random.random((12, 10))
scores2[scores2 < 0.75] = 0  # make "sparse"
sparsestack = StackedSparseArray(12, 10)
sparsestack.add_dense_matrix(scores1, "scores_1")

# Add second scores and filter
sparsestack.add_dense_matrix(scores2, "scores_2", join_type="left")

# Scores can be accessed using (limited) slicing capabilities
sparsestack[3, 4]  # => scores_1 and scores_2 at position row=3, col=4
sparsestack[3, :]  # => tuple with row, col, scores for all entries in row=3
sparsestack[:, 2]  # => tuple with row, col, scores for all entries in col=2
sparsestack[3, :, 0]  # => tuple with row, col, scores_1 for all entries in row=3
sparsestack[3, :, "scores_1"]  # => same as the one before

# Scores can also be converted to a dense numpy array:
scores2_after_merge = sparsestack.to_array("scores_2")

Adding data to a sparsestack-array

Sparsestack provides three options to add data to a new layer.

  1. .add_dense_matrix(input_array) Can be used to add all none-zero elements of input_array to the sparsestack. Depending on the chosen join_type either all such values will be added (join_type="outer" or join_type="right"), or only those which are already present in underlying layers ("left" or "inner" join).
  2. .add_sparse_matrix(input_coo_matrix) This method will expect a COO-style matrix (e.g. scipy) which has attributes .row, .col and .data. The join type can again be specified using join_type.
  3. .add_sparse_data(row, col, data) This essentially does the same as .add_sparse_matrix(input_coo_matrix) but might in some cases be a bit more flexible because row, col and data are separate input arguments.

Accessing data from sparsestack-array

The collected sparse data can be accessed in multiple ways.

  1. Slicing. sparsestack allows multiple types of slicing (see also code example above).
sparsestack[3, 4]  # => tuple with all scores at position row=3, col=4
sparsestack[3, :]  # => tuple with row, col, scores for all entries in row=3
sparsestack[:, 2]  # => tuple with row, col, scores for all entries in col=2
sparsestack[3, :, 0]  # => tuple with row, col, scores_1 for all entries in row=3
sparsestack[3, :, "scores_1"]  # => same as the one before
  1. .to_array() Creates and returns a dense numpy array of size .shape. Can also be used to create a dense numpy array of only a single layer when used like .to_array(name="layerX").
    Carefull: Obviously by converting to a dense array, the sparse nature will be lost and all empty positions in the stack will be filled with zeros.
  2. .to_coo(name="layerX") Returns a scipy sparse COO-matrix of the specified layer.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

sparsestack-0.6.4.tar.gz (9.8 kB view details)

Uploaded Source

Built Distribution

sparsestack-0.6.4-py3-none-any.whl (9.8 kB view details)

Uploaded Python 3

File details

Details for the file sparsestack-0.6.4.tar.gz.

File metadata

  • Download URL: sparsestack-0.6.4.tar.gz
  • Upload date:
  • Size: 9.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.8.4 CPython/3.11.0 Linux/6.5.0-1025-azure

File hashes

Hashes for sparsestack-0.6.4.tar.gz
Algorithm Hash digest
SHA256 28f93bae1ab9aeec079d77020afd7859a6b3ad78f754be5e1f81feaad9528b6e
MD5 4543ce67458a29912bd69bb41b6df7a3
BLAKE2b-256 f29945ac941792c04563d9e682a37f7f210ef300883cc10adb47af9667319a11

See more details on using hashes here.

File details

Details for the file sparsestack-0.6.4-py3-none-any.whl.

File metadata

  • Download URL: sparsestack-0.6.4-py3-none-any.whl
  • Upload date:
  • Size: 9.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.8.4 CPython/3.11.0 Linux/6.5.0-1025-azure

File hashes

Hashes for sparsestack-0.6.4-py3-none-any.whl
Algorithm Hash digest
SHA256 29274482d7f1b40d52cd06c56c48314823ca3b3de5f13e1ca2d87a9cb3135b4b
MD5 6e999aefae0e8a0473449ed9d9543d4b
BLAKE2b-256 532688a05ac1d30d70a8e11ee81b86a094c64e828641c8609179b4afee60433d

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page