Scipy sparse matrix in HDF5.
Project description
Please visit the Github repository for more information.
h5sparse
Scipy sparse matrix in HDF5.
Installation
pip install h5sparse
Testing
for single environment:
python setup.py test
for all environments:
tox
Examples
Create dataset
In [1]: import scipy.sparse as ss ...: import h5sparse ...: import numpy as np ...: In [2]: sparse_matrix = ss.csr_matrix([[0, 1, 0], ...: [0, 0, 1], ...: [0, 0, 0], ...: [1, 1, 0]], ...: dtype=np.float64) In [3]: # create dataset from scipy sparse matrix ...: with h5sparse.File("test.h5") as h5f: ...: h5f.create_dataset('sparse/matrix', data=sparse_matrix) In [4]: # you can also create dataset from another dataset ...: with h5sparse.File("test.h5") as h5f: ...: h5f.create_dataset('sparse/matrix2', data=h5f['sparse/matrix']) In [5]: # you can also create dataset using the formats that original h5py accepts ...: with h5sparse.File("test.h5") as h5f: ...: h5f.create_dataset('sparse/matrix3', data=[1,2,3])
Read dataset
In [6]: h5f = h5sparse.File("test.h5") In [7]: h5f['sparse/matrix'][1:3] Out[7]: <2x3 sparse matrix of type '<class 'numpy.float64'>' with 1 stored elements in Compressed Sparse Row format> In [8]: h5f['sparse/matrix'][1:3].toarray() Out[8]: array([[ 0., 0., 1.], [ 0., 0., 0.]]) In [9]: h5f['sparse']['matrix'][1:3].toarray() Out[9]: array([[ 0., 0., 1.], [ 0., 0., 0.]]) In [10]: h5f['sparse']['matrix'][2:].toarray() Out[10]: array([[ 0., 0., 0.], [ 1., 1., 0.]]) In [11]: h5f['sparse']['matrix'][:2].toarray() Out[11]: array([[ 0., 1., 0.], [ 0., 0., 1.]]) In [12]: h5f['sparse']['matrix'][-2:].toarray() Out[12]: array([[ 0., 0., 0.], [ 1., 1., 0.]]) In [13]: h5f['sparse']['matrix'][:-2].toarray() Out[13]: array([[ 0., 1., 0.], [ 0., 0., 1.]]) In [14]: h5f['sparse']['matrix'][()].toarray() Out[14]: array([[ 0., 1., 0.], [ 0., 0., 1.], [ 0., 0., 0.], [ 1., 1., 0.]]) In [15]: import h5py In [16]: h5py_h5f = h5py.File("test.h5") In [17]: h5sparse.Group(h5py_h5f.id)['sparse/matrix'][()] Out[17]: <4x3 sparse matrix of type '<class 'numpy.float64'>' with 4 stored elements in Compressed Sparse Row format> In [18]: h5sparse.Group(h5py_h5f['sparse'].id)['matrix'][()] Out[18]: <4x3 sparse matrix of type '<class 'numpy.float64'>' with 4 stored elements in Compressed Sparse Row format> In [19]: h5sparse.Dataset(h5py_h5f['sparse/matrix'])[()] Out[19]: <4x3 sparse matrix of type '<class 'numpy.float64'>' with 4 stored elements in Compressed Sparse Row format>
Append dataset
In [20]: to_append = ss.csr_matrix([[0, 1, 1], ...: [1, 0, 0]], ...: dtype=np.float64) In [21]: h5f.create_dataset('matrix', data=sparse_matrix, chunks=(100000,), ...: maxshape=(None,)) In [22]: h5f['matrix'].append(to_append) In [23]: h5f['matrix'][()] Out[23]: <6x3 sparse matrix of type '<class 'numpy.float64'>' with 7 stored elements in Compressed Sparse Row format> In [24]: h5f['matrix'][()].toarray() Out[24]: array([[ 0., 1., 0.], [ 0., 0., 1.], [ 0., 0., 0.], [ 1., 1., 0.], [ 0., 1., 1.], [ 1., 0., 0.]])
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Filename, size | File type | Python version | Upload date | Hashes |
---|---|---|---|---|
Filename, size h5sparse-0.1.0-py2.py3-none-any.whl (6.1 kB) | File type Wheel | Python version py2.py3 | Upload date | Hashes View |
Filename, size h5sparse-0.1.0.tar.gz (5.3 kB) | File type Source | Python version None | Upload date | Hashes View |
Close
Hashes for h5sparse-0.1.0-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 03595909c9fd4e895c4b3f12ca7a1e7f95a5cb3610f07d862d7f59612e0302fa |
|
MD5 | 552d4561859332b7faa88cd185064a30 |
|
BLAKE2-256 | 40ee8be7e1ba3844dc45e99e648d22fe214c8391c9e26245942221fea72a7e30 |