h5sparse

Please visit the Github repository for more information.

h5sparse

https://img.shields.io/pypi/l/h5sparse.svg

Scipy sparse matrix in HDF5.

Installation

pip install h5sparse

Testing

for single environment:
```
python setup.py test
```
for all environments:
```
tox
```

Examples

Create dataset

In [1]: import scipy.sparse as ss
   ...: import h5sparse
   ...: import numpy as np
   ...:

In [2]: sparse_matrix = ss.csr_matrix([[0, 1, 0],
   ...:                                [0, 0, 1],
   ...:                                [0, 0, 0],
   ...:                                [1, 1, 0]],
   ...:                               dtype=np.float64)

In [3]: # create dataset from scipy sparse matrix
   ...: with h5sparse.File("test.h5") as h5f:
   ...:     h5f.create_dataset('sparse/matrix', data=sparse_matrix)

In [4]: # you can also create dataset from another dataset
   ...: with h5sparse.File("test.h5") as h5f:
   ...:     h5f.create_dataset('sparse/matrix2', data=h5f['sparse/matrix'])

In [5]: # you can also create dataset using the formats that original h5py accepts
   ...: with h5sparse.File("test.h5") as h5f:
   ...:     h5f.create_dataset('sparse/matrix3', data=[1,2,3])

Read dataset

In [6]: h5f = h5sparse.File("test.h5")

In [7]: h5f['sparse/matrix'][1:3]
Out[7]:
<2x3 sparse matrix of type '<class 'numpy.float64'>'
        with 1 stored elements in Compressed Sparse Row format>

In [8]: h5f['sparse/matrix'][1:3].toarray()
Out[8]:
array([[ 0.,  0.,  1.],
       [ 0.,  0.,  0.]])

In [9]: h5f['sparse']['matrix'][1:3].toarray()
Out[9]:
array([[ 0.,  0.,  1.],
       [ 0.,  0.,  0.]])

In [10]: h5f['sparse']['matrix'][2:].toarray()
Out[10]:
array([[ 0.,  0.,  0.],
       [ 1.,  1.,  0.]])

In [11]: h5f['sparse']['matrix'][:2].toarray()
Out[11]:
array([[ 0.,  1.,  0.],
       [ 0.,  0.,  1.]])

In [12]: h5f['sparse']['matrix'][-2:].toarray()
Out[12]:
array([[ 0.,  0.,  0.],
       [ 1.,  1.,  0.]])

In [13]: h5f['sparse']['matrix'][:-2].toarray()
Out[13]:
array([[ 0.,  1.,  0.],
       [ 0.,  0.,  1.]])

In [14]: h5f['sparse']['matrix'][()].toarray()
Out[14]:
array([[ 0.,  1.,  0.],
       [ 0.,  0.,  1.],
       [ 0.,  0.,  0.],
       [ 1.,  1.,  0.]])

In [15]: import h5py

In [16]: h5py_h5f = h5py.File("test.h5")

In [17]: h5sparse.Group(h5py_h5f.id)['sparse/matrix'][()]
Out[17]:
<4x3 sparse matrix of type '<class 'numpy.float64'>'
        with 4 stored elements in Compressed Sparse Row format>

In [18]: h5sparse.Group(h5py_h5f['sparse'].id)['matrix'][()]
Out[18]:
<4x3 sparse matrix of type '<class 'numpy.float64'>'
        with 4 stored elements in Compressed Sparse Row format>

In [19]: h5sparse.Dataset(h5py_h5f['sparse/matrix'])[()]
Out[19]:
<4x3 sparse matrix of type '<class 'numpy.float64'>'
        with 4 stored elements in Compressed Sparse Row format>

Append dataset

In [20]: to_append = ss.csr_matrix([[0, 1, 1],
    ...:                            [1, 0, 0]],
    ...:                           dtype=np.float64)

In [21]: h5f.create_dataset('matrix', data=sparse_matrix, chunks=(100000,),
    ...:                    maxshape=(None,))

In [22]: h5f['matrix'].append(to_append)

In [23]: h5f['matrix'][()]
Out[23]:
<6x3 sparse matrix of type '<class 'numpy.float64'>'
        with 7 stored elements in Compressed Sparse Row format>

In [24]: h5f['matrix'][()].toarray()
Out[24]:
array([[ 0.,  1.,  0.],
       [ 0.,  0.,  1.],
       [ 0.,  0.,  0.],
       [ 1.,  1.,  0.],
       [ 0.,  1.,  1.],
       [ 1.,  0.,  0.]])

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

h5sparse-0.1.0.tar.gz (5.3 kB view details)

Uploaded Feb 24, 2019 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

h5sparse-0.1.0-py2.py3-none-any.whl (6.1 kB view details)

Uploaded Feb 24, 2019 Python 2Python 3

File details

Details for the file h5sparse-0.1.0.tar.gz.

File metadata

Download URL: h5sparse-0.1.0.tar.gz
Upload date: Feb 24, 2019
Size: 5.3 kB
Tags: Source
Uploaded using Trusted Publishing? No
Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.7.2

File hashes

Hashes for h5sparse-0.1.0.tar.gz
Algorithm	Hash digest
SHA256	`b7e6fbfa2f2f61adaa169a3a0a52bc6f2178f5daff8fa6d972a8dcaa7f3cc8f7`
MD5	`aa9b945fadade9fde9952be2b462af6e`
BLAKE2b-256	`a45cf4b3d34076bbfe863d36fa1c503a95b19a72caf2de23bff3c4ad770a5297`

See more details on using hashes here.

File details

Details for the file h5sparse-0.1.0-py2.py3-none-any.whl.

File metadata

Download URL: h5sparse-0.1.0-py2.py3-none-any.whl
Upload date: Feb 24, 2019
Size: 6.1 kB
Tags: Python 2, Python 3
Uploaded using Trusted Publishing? No
Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.7.2

File hashes

Hashes for h5sparse-0.1.0-py2.py3-none-any.whl
Algorithm	Hash digest
SHA256	`03595909c9fd4e895c4b3f12ca7a1e7f95a5cb3610f07d862d7f59612e0302fa`
MD5	`552d4561859332b7faa88cd185064a30`
BLAKE2b-256	`40ee8be7e1ba3844dc45e99e648d22fe214c8391c9e26245942221fea72a7e30`