All your matrix representations belong here!
Project description
Python bindings for tatami
Overview
The mattress package implements Python bindings to the tatami C++ library for matrix representations. Downstream packages can use mattress to develop C++ extensions that are interoperable with many different matrix classes, e.g., dense, sparse, delayed or file-backed. mattress is inspired by the beachmat Bioconductor package, which does the same thing for R packages.
Instructions
mattress is published to PyPI, so installation is simple:
pip install mattress
mattress is intended for Python package developers writing C++ extensions that operate on matrices.
- Add
mattress.includes()
to theinclude_dirs=
of yourExtension()
definition insetup.py
. This will give you access to the various tatami headers to compile your C++ code. - Add
#include "Mattress.h"
to your C++ source files. This defines aMattress
class where theptr
member is a pointer to a tatami matrix. Python-visible C++ functions should expect to take aMattress*
or equivalent address (e.g.,uintptr_t
), after which theptr
should be extracted for use in tatami-compatible functions. - Call
mattress.tatamize()
on Python matrix objects within each of your functions that call tatami C++ code. This will wrap the Python matrix in a tatami-compatible C++ representation for use in the C++ code. The pointer to the C++ instance can be accessed through theptr
property of the returned object, which can then be passed to C++ code as anuintptr_t
to aMattress
instance.
So, for example, we can write ctypes bindings like:
#include "Mattress.h"
extern "C" {
int do_something_interesting(const void* mat) {
return reinterpret_cast<const Mattress*>(mat)->ptr->nrow();
}
}
Which we can subsequently call like:
import mattress
import ctypes as ct
lib = ct.CDLL("compiled.so")
lib.do_something_interesting.restype = ct.c_int
lib.do_something_interesting.argtypes = [ ct.c_void_p ]
def do_something_interesting(x):
mat = mattress.tatamize(x)
return do_something_interesting(x.ptr)
Of course, any FFI that accepts a pointer address can be used here.
Supported matrices
Dense numpy matrices of varying numeric type:
import numpy as np
from mattress import tatamize
x = np.random.rand(1000, 100)
tatamat = tatamize(x)
ix = (x * 100).astype(np.uint16)
tatamat2 = tatamize(ix)
Compressed sparse matrices from scipy with varying index/data types:
from scipy import sparse as sp
from mattress import tatamize
xc = sp.random(100, 20, format="csc")
tatamat = tatamize(xc)
xr = sp.random(100, 20, format="csc", dtype=np.uint8)
tatamat2 = tatamize(xr)
Delayed arrays from the delayedarray package:
from delayedarray import DelayedArray
from scipy import sparse as sp
from mattress import tatamize
import numpy
xd = DelayedArray(sp.random(100, 20, format="csc"))
xd = numpy.log1p(xd * 5)
tatada = tatamize(xd)
To be added:
- File-backed matrices from the FileBackedArray package, including HDF5 and TileDB.
- Arbitrary Python matrices?
Utility methods
The TatamiNumericPointer
instance returned by tatamize()
provides a few Python-visible methods for querying the C++ matrix.
tatamat.nrow() // number of rows
tatamat.column(1) // contents of column 1
tatamat.sparse() // whether the matrix is sparse.
It also has a few methods for computing common statistics:
tatamat.row_sums()
tatamat.column_variances(num_threads = 2)
grouping = [i%3 for i in range(tatamat.ncol())]
tatamat.row_medians_by_group(grouping)
tatamat.row_nan_counts()
tatamat.column_ranges()
These are mostly intended for non-intensive work or testing/debugging. It is expected that any serious computation should be performed by iterating over the matrix in C++.
Operating on an existing pointer
If we already have a TatamiNumericPointer
, we can easily apply additional operations by wrapping it in the relevant delayedarray layers and calling tatamize()
afterwards.
For example, if we want to add a scalar, we might do:
from delayedarray import DelayedArray
from mattress import tatamize
import numpy
x = numpy.random.rand(1000, 10)
tatamat = tatamize(x)
wrapped = DelayedArray(tatamat) + 1
tatamat2 = tatamize(wrapped)
This avoids relying on x
and is more efficient as it re-uses the TatamiNumericPointer
generated from x
.
Developer Notes
Build the shared object file:
python setup.py build_ext --inplace
For quick testing, we usually do:
pytest
For more complex testing, we do:
python setup.py build_ext --inplace && tox
To rebuild the ctypes bindings with cpptypes:
cpptypes src/mattress/lib --py src/mattress/_cpphelpers.py --cpp src/mattress/lib/bindings.cpp --dll _core
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for mattress-0.2.0-cp311-cp311-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1e05e24e153a18e576e77faf70ec0c43286904085ea22dde5aa3b547a354bca8 |
|
MD5 | 16498ab7e623ecf80ab17154f41054b2 |
|
BLAKE2b-256 | 3c4b0184a38ae58effbe87635a31ee8ebedeb8ee1a8d8f873887e7225115e70c |
Hashes for mattress-0.2.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 63ed6edfc715c498b8ccef0e3a77ed76ba3b52846d28942baced1be630d74b71 |
|
MD5 | 8e8d65470fbede3891a8a870b27f53d8 |
|
BLAKE2b-256 | f857e89903d40ca15deea4d3bf9633ca07e6cb3825f832887e7900b24a5ebfed |
Hashes for mattress-0.2.0-cp311-cp311-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 168baca27100daa0da6044246f942318b066b7ceaafb9230740ebd6090629e06 |
|
MD5 | cb54d244b8b68628ea87ca0384149fb5 |
|
BLAKE2b-256 | 320c270573508e7a23bb66dad49e20e7c200be78fcc078056e6a209ac7f1c0c7 |
Hashes for mattress-0.2.0-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f8bfaf95ddd42101c0c13b35888bca48a44deebd240572c5fe92755438e55c6e |
|
MD5 | ede597c5f41917a90d6de9c47eab9ad9 |
|
BLAKE2b-256 | a4f7af08fb10460277d60c34a70519d655ac974f8fb0072457af802ce5093a4d |
Hashes for mattress-0.2.0-cp310-cp310-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 16e5517c1ff68893e8d408249194b95c522d872ce13891e01ebc5d0f17baab26 |
|
MD5 | 0033f36a8ad734640a8cd428746cea99 |
|
BLAKE2b-256 | d1d0bc5db9236b9ba7c592ff4b4f1fb7b0e00ac846fb74fc31db9c52efa84304 |
Hashes for mattress-0.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dba47bd44f14e173d3b2df2405737434774cfda7f2b6034e3943e190431c21d4 |
|
MD5 | 4b027c22dee16c3887d502d675bea160 |
|
BLAKE2b-256 | ce8f9430b40ff27cc8ff8c6da20655ee2364d9c368b4239d3d813c2f7c0c88c5 |
Hashes for mattress-0.2.0-cp310-cp310-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f33c39b067d36542832873f1b8f7da8cb76a5124cb89311cd04f35a0da1179d8 |
|
MD5 | 852326c3265cb261c075538cbce49190 |
|
BLAKE2b-256 | c710396467424485cf90a86c6b649433d6d428218a5feb9e475fa53e10f07063 |
Hashes for mattress-0.2.0-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bbf9a324dcaf21657f7196ebfdd7acf4b3b385ac1b0634469e31af64f60aa5fa |
|
MD5 | da9b8933cd001df352151ee3ec96d968 |
|
BLAKE2b-256 | cb127ca5ea4be6bd62f5d7883ce28bc48ba44b8f442d9bb37be1bd47a3af656f |
Hashes for mattress-0.2.0-cp39-cp39-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f82f2b1b7cccbfb111ee29b0f9e6a9eb7fb091f00b26e999607179b9b1dbf668 |
|
MD5 | fe95d41bb9db1ae178eec8034c940181 |
|
BLAKE2b-256 | c84fa334056b9837f56678d9aed867b02ccdc7c64488843bb0ce895b818c8e5f |
Hashes for mattress-0.2.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | be156054a055d92146083a444fd956baeba521e06629f891ab373b0a387fda07 |
|
MD5 | 3f3b13b0703d8c826b48903c2a3a58c3 |
|
BLAKE2b-256 | cbc45851357a3ca184e6ccbfb9854e914412ffefc24c66568a193bff831b1398 |
Hashes for mattress-0.2.0-cp39-cp39-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d4d42adfe90d2e88920332dda4a6613a9d16a7815949787b25a496ac62726cf8 |
|
MD5 | fd73247adb5c709ade92968d2053541a |
|
BLAKE2b-256 | ace27f17551fe35d60e86a94c61d529553d702595caa25d2034c6fdc0366a4ff |
Hashes for mattress-0.2.0-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1e3504c86f8ce747ad8c3c2ea84852e8b3518ecc8463f969d1188e9275f5ff28 |
|
MD5 | ab82c0193c4533b3475788f7e9269237 |
|
BLAKE2b-256 | 691240a0f878e8a75ee41d7b95f5098e9e5d686830df7dd8ee6d85918cdf1406 |
Hashes for mattress-0.2.0-cp38-cp38-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 88142e1d07fff021da1c52dfe927ccf096034d97ee4560d151211aee47637003 |
|
MD5 | c6c1efe08e80e1649df7d24398b7c248 |
|
BLAKE2b-256 | 04d4b93ce43f614b453eac9527f3a262b6dd36ac1c23fdee586484ec5e5df8de |
Hashes for mattress-0.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5be34a854b05e9ce09e0e06d14c99f9255e4aeb48f9d86fc3619bbe998c3abf7 |
|
MD5 | 8bdd749ed3fcdb47fd47c4bd8fab5432 |
|
BLAKE2b-256 | ed57cbe8b13caa47e45994945f9646dfc3e904e587c20f11942a86b4aaf57ec4 |
Hashes for mattress-0.2.0-cp38-cp38-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f2e8bade28bfdda44a88a3a4ed2b53844de0efa2c69ebaea8332794b7986b46e |
|
MD5 | 53faf894236a7f750e3937c36cbabeb3 |
|
BLAKE2b-256 | 24802c3b079aa0484e254f1908ee37665eefac644440cbb97798efb6a2b09b13 |
Hashes for mattress-0.2.0-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f8f0ebf677def59587e3993564d80b51ed5126b7b43ca8c0626f4a7593a31a2a |
|
MD5 | 0223ab324edc0d4e274f38c1b323966f |
|
BLAKE2b-256 | 7ed02ed248870fd704bfb9832af4a88b1082e1c3468ee82ee38af98d67758212 |