Skip to main content

Scipy sparse matrix in HDF5.

Project description

Please visit the Github repository for more information.

h5sparse

https://img.shields.io/travis/appier/h5sparse/master.svg https://img.shields.io/pypi/v/h5sparse.svg https://img.shields.io/pypi/l/h5sparse.svg

Scipy sparse matrix in HDF5.

Installation

pip install h5sparse

Testing

  • for single environment:

    python setup.py test
    
  • for all environments:

    tox
    

Examples

Create dataset

In [1]: import scipy.sparse as ss
   ...: import h5sparse
   ...: import numpy as np
   ...:

In [2]: sparse_matrix = ss.csr_matrix([[0, 1, 0],
   ...:                                [0, 0, 1],
   ...:                                [0, 0, 0],
   ...:                                [1, 1, 0]],
   ...:                               dtype=np.float64)

In [3]: # create dataset from scipy sparse matrix
   ...: with h5sparse.File("test.h5") as h5f:
   ...:     h5f.create_dataset('sparse/matrix', data=sparse_matrix)

In [4]: # you can also create dataset from another dataset
   ...: with h5sparse.File("test.h5") as h5f:
   ...:     h5f.create_dataset('sparse/matrix2', data=h5f['sparse/matrix'])

In [5]: # you can also create dataset using the formats that original h5py accepts
   ...: with h5sparse.File("test.h5") as h5f:
   ...:     h5f.create_dataset('sparse/matrix3', data=[1,2,3])

Read dataset

In [6]: h5f = h5sparse.File("test.h5")

In [7]: h5f['sparse/matrix'][1:3]
Out[7]:
<2x3 sparse matrix of type '<class 'numpy.float64'>'
        with 1 stored elements in Compressed Sparse Row format>

In [8]: h5f['sparse/matrix'][1:3].toarray()
Out[8]:
array([[ 0.,  0.,  1.],
       [ 0.,  0.,  0.]])

In [9]: h5f['sparse']['matrix'][1:3].toarray()
Out[9]:
array([[ 0.,  0.,  1.],
       [ 0.,  0.,  0.]])

In [10]: h5f['sparse']['matrix'][2:].toarray()
Out[10]:
array([[ 0.,  0.,  0.],
       [ 1.,  1.,  0.]])

In [11]: h5f['sparse']['matrix'][:2].toarray()
Out[11]:
array([[ 0.,  1.,  0.],
       [ 0.,  0.,  1.]])

In [12]: h5f['sparse']['matrix'][-2:].toarray()
Out[12]:
array([[ 0.,  0.,  0.],
       [ 1.,  1.,  0.]])

In [13]: h5f['sparse']['matrix'][:-2].toarray()
Out[13]:
array([[ 0.,  1.,  0.],
       [ 0.,  0.,  1.]])

In [14]: h5f['sparse']['matrix'][()].toarray()
Out[14]:
array([[ 0.,  1.,  0.],
       [ 0.,  0.,  1.],
       [ 0.,  0.,  0.],
       [ 1.,  1.,  0.]])

In [15]: import h5py

In [16]: h5py_h5f = h5py.File("test.h5")

In [17]: h5sparse.Group(h5py_h5f.id)['sparse/matrix'][()]
Out[17]:
<4x3 sparse matrix of type '<class 'numpy.float64'>'
        with 4 stored elements in Compressed Sparse Row format>

In [18]: h5sparse.Group(h5py_h5f['sparse'].id)['matrix'][()]
Out[18]:
<4x3 sparse matrix of type '<class 'numpy.float64'>'
        with 4 stored elements in Compressed Sparse Row format>

In [19]: h5sparse.Dataset(h5py_h5f['sparse/matrix'])[()]
Out[19]:
<4x3 sparse matrix of type '<class 'numpy.float64'>'
        with 4 stored elements in Compressed Sparse Row format>

Append dataset

In [20]: to_append = ss.csr_matrix([[0, 1, 1],
    ...:                            [1, 0, 0]],
    ...:                           dtype=np.float64)

In [21]: h5f.create_dataset('matrix', data=sparse_matrix, chunks=(100000,),
    ...:                    maxshape=(None,))

In [22]: h5f['matrix'].append(to_append)

In [23]: h5f['matrix'][()]
Out[23]:
<6x3 sparse matrix of type '<class 'numpy.float64'>'
        with 7 stored elements in Compressed Sparse Row format>

In [24]: h5f['matrix'][()].toarray()
Out[24]:
array([[ 0.,  1.,  0.],
       [ 0.,  0.,  1.],
       [ 0.,  0.,  0.],
       [ 1.,  1.,  0.],
       [ 0.,  1.,  1.],
       [ 1.,  0.,  0.]])

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

h5sparse-0.1.0.tar.gz (5.3 kB view hashes)

Uploaded source

Built Distribution

h5sparse-0.1.0-py2.py3-none-any.whl (6.1 kB view hashes)

Uploaded py2 py3

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page