Skip to main content

All your matrix representations belong here!

Project description

PyPI-Server Monthly Downloads Unit tests

Python bindings for tatami

Overview

The mattress package implements Python bindings to the tatami C++ library for matrix representations. Downstream packages can use mattress to develop C++ extensions that are interoperable with many different matrix classes, e.g., dense, sparse, delayed or file-backed. mattress is inspired by the beachmat Bioconductor package, which does the same thing for R packages.

Instructions

mattress is published to PyPI, so installation is simple:

pip install mattress

mattress is intended for Python package developers writing C++ extensions that operate on matrices.

  1. Add mattress.includes() to the include_dirs= of your Extension() definition in setup.py. This will give you access to the various tatami headers to compile your C++ code.
  2. Add #include "Mattress.h" to your C++ source files. This defines a Mattress class where the ptr member is a pointer to a tatami matrix. Python-visible C++ functions should expect to take a Mattress* or equivalent address (e.g., uintptr_t), after which the ptr should be extracted for use in tatami-compatible functions.
  3. Call mattress.tatamize() on Python matrix objects within each of your functions that call tatami C++ code. This will wrap the Python matrix in a tatami-compatible C++ representation for use in the C++ code. The pointer to the C++ instance can be accessed through the ptr property of the returned object, which can then be passed to C++ code as an uintptr_t to a Mattress instance.

So, for example, we can write ctypes bindings like:

#include "Mattress.h"

extern "C" {

int do_something_interesting(const void* mat) {
    return reinterpret_cast<const Mattress*>(mat)->ptr->nrow();
}

}

Which we can subsequently call like:

import mattress

import ctypes as ct
lib = ct.CDLL("compiled.so")
lib.do_something_interesting.restype = ct.c_int
lib.do_something_interesting.argtypes = [ ct.c_void_p ]

def do_something_interesting(x):
    mat = mattress.tatamize(x)
    return do_something_interesting(x.ptr)

Of course, any FFI that accepts a pointer address can be used here.

Supported matrices

Dense numpy matrices of varying numeric type:

import numpy as np
from mattress import tatamize
x = np.random.rand(1000, 100)
tatamat = tatamize(x)

ix = (x * 100).astype(np.uint16)
tatamat2 = tatamize(ix)

Compressed sparse matrices from scipy with varying index/data types:

from scipy import sparse as sp
from mattress import tatamize

xc = sp.random(100, 20, format="csc")
tatamat = tatamize(xc)

xr = sp.random(100, 20, format="csc", dtype=np.uint8)
tatamat2 = tatamize(xr)

Delayed arrays from the delayedarray package:

from delayedarray import DelayedArray
from scipy import sparse as sp
from mattress import tatamize
import numpy

xd = DelayedArray(sp.random(100, 20, format="csc"))
xd = numpy.log1p(xd * 5)

tatada = tatamize(xd)

To be added:

  • File-backed matrices from the FileBackedArray package, including HDF5 and TileDB.
  • Arbitrary Python matrices?

Utility methods

The TatamiNumericPointer instance returned by tatamize() provides a few Python-visible methods for querying the C++ matrix.

tatamat.nrow() // number of rows
tatamat.column(1) // contents of column 1
tatamat.sparse() // whether the matrix is sparse.

It also has a few methods for computing common statistics:

tatamat.row_sums()
tatamat.column_variances(num_threads = 2)

grouping = [i%3 for i in range(tatamat.ncol())]
tatamat.row_medians_by_group(grouping)

tatamat.row_nan_counts()
tatamat.column_ranges()

These are mostly intended for non-intensive work or testing/debugging. It is expected that any serious computation should be performed by iterating over the matrix in C++.

Operating on an existing pointer

If we already have a TatamiNumericPointer, we can easily apply additional operations by wrapping it in the relevant delayedarray layers and calling tatamize() afterwards. For example, if we want to add a scalar, we might do:

from delayedarray import DelayedArray
from mattress import tatamize
import numpy

x = numpy.random.rand(1000, 10)
tatamat = tatamize(x)

wrapped = DelayedArray(tatamat) + 1
tatamat2 = tatamize(wrapped)

This avoids relying on x and is more efficient as it re-uses the TatamiNumericPointer generated from x.

Developer Notes

Build the shared object file:

python setup.py build_ext --inplace

For quick testing, we usually do:

pytest

For more complex testing, we do:

python setup.py build_ext --inplace && tox

To rebuild the ctypes bindings with cpptypes:

cpptypes src/mattress/lib --py src/mattress/_cpphelpers.py --cpp src/mattress/lib/bindings.cpp --dll _core

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

mattress-0.2.0.tar.gz (41.6 kB view details)

Uploaded Source

Built Distributions

mattress-0.2.0-cp311-cp311-musllinux_1_1_x86_64.whl (54.6 MB view details)

Uploaded CPython 3.11 musllinux: musl 1.1+ x86-64

mattress-0.2.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (54.8 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

mattress-0.2.0-cp311-cp311-macosx_11_0_arm64.whl (2.6 MB view details)

Uploaded CPython 3.11 macOS 11.0+ ARM64

mattress-0.2.0-cp311-cp311-macosx_10_9_x86_64.whl (3.2 MB view details)

Uploaded CPython 3.11 macOS 10.9+ x86-64

mattress-0.2.0-cp310-cp310-musllinux_1_1_x86_64.whl (54.6 MB view details)

Uploaded CPython 3.10 musllinux: musl 1.1+ x86-64

mattress-0.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (54.8 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

mattress-0.2.0-cp310-cp310-macosx_11_0_arm64.whl (2.6 MB view details)

Uploaded CPython 3.10 macOS 11.0+ ARM64

mattress-0.2.0-cp310-cp310-macosx_10_9_x86_64.whl (3.2 MB view details)

Uploaded CPython 3.10 macOS 10.9+ x86-64

mattress-0.2.0-cp39-cp39-musllinux_1_1_x86_64.whl (54.6 MB view details)

Uploaded CPython 3.9 musllinux: musl 1.1+ x86-64

mattress-0.2.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (54.8 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

mattress-0.2.0-cp39-cp39-macosx_11_0_arm64.whl (2.6 MB view details)

Uploaded CPython 3.9 macOS 11.0+ ARM64

mattress-0.2.0-cp39-cp39-macosx_10_9_x86_64.whl (3.2 MB view details)

Uploaded CPython 3.9 macOS 10.9+ x86-64

mattress-0.2.0-cp38-cp38-musllinux_1_1_x86_64.whl (54.6 MB view details)

Uploaded CPython 3.8 musllinux: musl 1.1+ x86-64

mattress-0.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (54.8 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

mattress-0.2.0-cp38-cp38-macosx_11_0_arm64.whl (2.6 MB view details)

Uploaded CPython 3.8 macOS 11.0+ ARM64

mattress-0.2.0-cp38-cp38-macosx_10_9_x86_64.whl (3.2 MB view details)

Uploaded CPython 3.8 macOS 10.9+ x86-64

File details

Details for the file mattress-0.2.0.tar.gz.

File metadata

  • Download URL: mattress-0.2.0.tar.gz
  • Upload date:
  • Size: 41.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.11.9

File hashes

Hashes for mattress-0.2.0.tar.gz
Algorithm Hash digest
SHA256 444fb8542e18ee915612cfdb609a1667ffef93113663d2dbdfb67c5ed287cca9
MD5 c9a2a0338fb6bb4e395d6ef2eb9f6f87
BLAKE2b-256 c16827dd459eef77f3db026d6c394d2765fdbaee9e177b3858a88b7c46bd244f

See more details on using hashes here.

File details

Details for the file mattress-0.2.0-cp311-cp311-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for mattress-0.2.0-cp311-cp311-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 1e05e24e153a18e576e77faf70ec0c43286904085ea22dde5aa3b547a354bca8
MD5 16498ab7e623ecf80ab17154f41054b2
BLAKE2b-256 3c4b0184a38ae58effbe87635a31ee8ebedeb8ee1a8d8f873887e7225115e70c

See more details on using hashes here.

File details

Details for the file mattress-0.2.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for mattress-0.2.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 63ed6edfc715c498b8ccef0e3a77ed76ba3b52846d28942baced1be630d74b71
MD5 8e8d65470fbede3891a8a870b27f53d8
BLAKE2b-256 f857e89903d40ca15deea4d3bf9633ca07e6cb3825f832887e7900b24a5ebfed

See more details on using hashes here.

File details

Details for the file mattress-0.2.0-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for mattress-0.2.0-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 168baca27100daa0da6044246f942318b066b7ceaafb9230740ebd6090629e06
MD5 cb54d244b8b68628ea87ca0384149fb5
BLAKE2b-256 320c270573508e7a23bb66dad49e20e7c200be78fcc078056e6a209ac7f1c0c7

See more details on using hashes here.

File details

Details for the file mattress-0.2.0-cp311-cp311-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for mattress-0.2.0-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 f8bfaf95ddd42101c0c13b35888bca48a44deebd240572c5fe92755438e55c6e
MD5 ede597c5f41917a90d6de9c47eab9ad9
BLAKE2b-256 a4f7af08fb10460277d60c34a70519d655ac974f8fb0072457af802ce5093a4d

See more details on using hashes here.

File details

Details for the file mattress-0.2.0-cp310-cp310-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for mattress-0.2.0-cp310-cp310-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 16e5517c1ff68893e8d408249194b95c522d872ce13891e01ebc5d0f17baab26
MD5 0033f36a8ad734640a8cd428746cea99
BLAKE2b-256 d1d0bc5db9236b9ba7c592ff4b4f1fb7b0e00ac846fb74fc31db9c52efa84304

See more details on using hashes here.

File details

Details for the file mattress-0.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for mattress-0.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 dba47bd44f14e173d3b2df2405737434774cfda7f2b6034e3943e190431c21d4
MD5 4b027c22dee16c3887d502d675bea160
BLAKE2b-256 ce8f9430b40ff27cc8ff8c6da20655ee2364d9c368b4239d3d813c2f7c0c88c5

See more details on using hashes here.

File details

Details for the file mattress-0.2.0-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for mattress-0.2.0-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 f33c39b067d36542832873f1b8f7da8cb76a5124cb89311cd04f35a0da1179d8
MD5 852326c3265cb261c075538cbce49190
BLAKE2b-256 c710396467424485cf90a86c6b649433d6d428218a5feb9e475fa53e10f07063

See more details on using hashes here.

File details

Details for the file mattress-0.2.0-cp310-cp310-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for mattress-0.2.0-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 bbf9a324dcaf21657f7196ebfdd7acf4b3b385ac1b0634469e31af64f60aa5fa
MD5 da9b8933cd001df352151ee3ec96d968
BLAKE2b-256 cb127ca5ea4be6bd62f5d7883ce28bc48ba44b8f442d9bb37be1bd47a3af656f

See more details on using hashes here.

File details

Details for the file mattress-0.2.0-cp39-cp39-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for mattress-0.2.0-cp39-cp39-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 f82f2b1b7cccbfb111ee29b0f9e6a9eb7fb091f00b26e999607179b9b1dbf668
MD5 fe95d41bb9db1ae178eec8034c940181
BLAKE2b-256 c84fa334056b9837f56678d9aed867b02ccdc7c64488843bb0ce895b818c8e5f

See more details on using hashes here.

File details

Details for the file mattress-0.2.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for mattress-0.2.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 be156054a055d92146083a444fd956baeba521e06629f891ab373b0a387fda07
MD5 3f3b13b0703d8c826b48903c2a3a58c3
BLAKE2b-256 cbc45851357a3ca184e6ccbfb9854e914412ffefc24c66568a193bff831b1398

See more details on using hashes here.

File details

Details for the file mattress-0.2.0-cp39-cp39-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for mattress-0.2.0-cp39-cp39-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 d4d42adfe90d2e88920332dda4a6613a9d16a7815949787b25a496ac62726cf8
MD5 fd73247adb5c709ade92968d2053541a
BLAKE2b-256 ace27f17551fe35d60e86a94c61d529553d702595caa25d2034c6fdc0366a4ff

See more details on using hashes here.

File details

Details for the file mattress-0.2.0-cp39-cp39-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for mattress-0.2.0-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 1e3504c86f8ce747ad8c3c2ea84852e8b3518ecc8463f969d1188e9275f5ff28
MD5 ab82c0193c4533b3475788f7e9269237
BLAKE2b-256 691240a0f878e8a75ee41d7b95f5098e9e5d686830df7dd8ee6d85918cdf1406

See more details on using hashes here.

File details

Details for the file mattress-0.2.0-cp38-cp38-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for mattress-0.2.0-cp38-cp38-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 88142e1d07fff021da1c52dfe927ccf096034d97ee4560d151211aee47637003
MD5 c6c1efe08e80e1649df7d24398b7c248
BLAKE2b-256 04d4b93ce43f614b453eac9527f3a262b6dd36ac1c23fdee586484ec5e5df8de

See more details on using hashes here.

File details

Details for the file mattress-0.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for mattress-0.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 5be34a854b05e9ce09e0e06d14c99f9255e4aeb48f9d86fc3619bbe998c3abf7
MD5 8bdd749ed3fcdb47fd47c4bd8fab5432
BLAKE2b-256 ed57cbe8b13caa47e45994945f9646dfc3e904e587c20f11942a86b4aaf57ec4

See more details on using hashes here.

File details

Details for the file mattress-0.2.0-cp38-cp38-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for mattress-0.2.0-cp38-cp38-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 f2e8bade28bfdda44a88a3a4ed2b53844de0efa2c69ebaea8332794b7986b46e
MD5 53faf894236a7f750e3937c36cbabeb3
BLAKE2b-256 24802c3b079aa0484e254f1908ee37665eefac644440cbb97798efb6a2b09b13

See more details on using hashes here.

File details

Details for the file mattress-0.2.0-cp38-cp38-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for mattress-0.2.0-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 f8f0ebf677def59587e3993564d80b51ed5126b7b43ca8c0626f4a7593a31a2a
MD5 0223ab324edc0d4e274f38c1b323966f
BLAKE2b-256 7ed02ed248870fd704bfb9832af4a88b1082e1c3468ee82ee38af98d67758212

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page