Skip to main content
GitHub Actions Workflow Status https://img.shields.io/pypi/v/h5sparse-tensor.svg https://img.shields.io/pypi/l/h5sparse-tensor.svg

Scipy sparse matrix in HDF5.

Installation

pip install h5sparse-tensor

Testing

  • for single environment:

    python setup.py test
  • for all environments:

    tox

Examples

Create dataset

In [1]: import scipy.sparse as ss
   ...: import h5sparse
   ...: import numpy as np
   ...:

In [2]: sparse_matrix = ss.csr_matrix([[0, 1, 0],
   ...:                                [0, 0, 1],
   ...:                                [0, 0, 0],
   ...:                                [1, 1, 0]],
   ...:                               dtype=np.float64)

In [3]: # create dataset from scipy sparse matrix
   ...: with h5sparse.File("test.h5") as h5f:
   ...:     h5f.create_dataset('sparse/matrix', data=sparse_matrix)

In [4]: # you can also create dataset from another dataset
   ...: with h5sparse.File("test.h5") as h5f:
   ...:     h5f.create_dataset('sparse/matrix2', data=h5f['sparse/matrix'])

In [5]: # you can also create dataset using the formats that original h5py accepts
   ...: with h5sparse.File("test.h5") as h5f:
   ...:     h5f.create_dataset('sparse/matrix3', data=[1,2,3])

Read dataset

In [6]: h5f = h5sparse.File("test.h5")

In [7]: h5f['sparse/matrix'][1:3]
Out[7]:
<2x3 sparse matrix of type '<class 'numpy.float64'>'
        with 1 stored elements in Compressed Sparse Row format>

In [8]: h5f['sparse/matrix'][1:3].toarray()
Out[8]:
array([[ 0.,  0.,  1.],
       [ 0.,  0.,  0.]])

In [9]: h5f['sparse']['matrix'][1:3].toarray()
Out[9]:
array([[ 0.,  0.,  1.],
       [ 0.,  0.,  0.]])

In [10]: h5f['sparse']['matrix'][2:].toarray()
Out[10]:
array([[ 0.,  0.,  0.],
       [ 1.,  1.,  0.]])

In [11]: h5f['sparse']['matrix'][:2].toarray()
Out[11]:
array([[ 0.,  1.,  0.],
       [ 0.,  0.,  1.]])

In [12]: h5f['sparse']['matrix'][-2:].toarray()
Out[12]:
array([[ 0.,  0.,  0.],
       [ 1.,  1.,  0.]])

In [13]: h5f['sparse']['matrix'][:-2].toarray()
Out[13]:
array([[ 0.,  1.,  0.],
       [ 0.,  0.,  1.]])

In [14]: h5f['sparse']['matrix'][()].toarray()
Out[14]:
array([[ 0.,  1.,  0.],
       [ 0.,  0.,  1.],
       [ 0.,  0.,  0.],
       [ 1.,  1.,  0.]])

In [15]: import h5py

In [16]: h5py_h5f = h5py.File("test.h5")

In [17]: h5sparse.Group(h5py_h5f.id)['sparse/matrix'][()]
Out[17]:
<4x3 sparse matrix of type '<class 'numpy.float64'>'
        with 4 stored elements in Compressed Sparse Row format>

In [18]: h5sparse.Group(h5py_h5f['sparse'].id)['matrix'][()]
Out[18]:
<4x3 sparse matrix of type '<class 'numpy.float64'>'
        with 4 stored elements in Compressed Sparse Row format>

In [19]: h5sparse.Dataset(h5py_h5f['sparse/matrix'])[()]
Out[19]:
<4x3 sparse matrix of type '<class 'numpy.float64'>'
        with 4 stored elements in Compressed Sparse Row format>

Append dataset

In [20]: to_append = ss.csr_matrix([[0, 1, 1],
    ...:                            [1, 0, 0]],
    ...:                           dtype=np.float64)

In [21]: h5f.create_dataset('matrix', data=sparse_matrix, chunks=(100000,),
    ...:                    maxshape=(None,))

In [22]: h5f['matrix'].append(to_append)

In [23]: h5f['matrix'][()]
Out[23]:
<6x3 sparse matrix of type '<class 'numpy.float64'>'
        with 7 stored elements in Compressed Sparse Row format>

In [24]: h5f['matrix'][()].toarray()
Out[24]:
array([[ 0.,  1.,  0.],
       [ 0.,  0.,  1.],
       [ 0.,  0.,  0.],
       [ 1.,  1.,  0.],
       [ 0.,  1.,  1.],
       [ 1.,  0.,  0.]])

Metadata

Release files for h5sparse-tensor 0.2.4

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for h5sparse-tensor 0.2.4
File Size Uploaded
h5sparse_tensor-0.2.4.tar.gz 6.7 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for h5sparse-tensor 0.2.4
File Interpreter ABI Platform
h5sparse_tensor-0.2.4-py2.py3-none-any.whl Python 2, Python 3 none any Details

Total release size: 12.3 kB

Release files / h5sparse_tensor-0.2.4.tar.gz

Download URL h5sparse_tensor-0.2.4.tar.gz
Size 6.7 kB
Tags Source
SHA-256 checksum
How to use checksums
de89325c5563100a6ea85b4d41fe4f318f11a86a5010798d41c99aca612910c3
BLAKE2b-256 checksum
How to use checksums
0bd71254a3981278c74c669eda033f4edb3c040ac29207cda1b68605043eb62e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.12.3

Release files / h5sparse_tensor-0.2.4-py2.py3-none-any.whl

Download URL h5sparse_tensor-0.2.4-py2.py3-none-any.whl
Size 5.6 kB
Tags Python 2 Python 3
SHA-256 checksum
How to use checksums
8c23b6b82dbb511d153c2f70eb433e18fa4c077b84cda2383e24a180a2304ddf
BLAKE2b-256 checksum
How to use checksums
5325cbd81f2b50877b599d481d17265b92adabd9f6f5fcebe28fa0be3d539888
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.12.3

Release history Release notifications | RSS feed

This release

0.2.4 This release

2 release files

0.2.3

2 release files

0.2.2

1 release file

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page