hdf5-tools
This git repository contains a python package with an H5 class to load and combine one or more HDF5 data files (or xarray datasets) with optional filters. The class will then export the combined data to an HDF5 file, file object, or xr.Dataset. This class is designed to be fast and safe on memory. This means that files of any size can be combined and saved even on a PC with low memory (unlike xarray).
Installation
Using pip:
pip install hdf5tools
Or using conda/mamba from conda-forge:
conda install -c conda-forge hdf5tools
Usage
Currently, only the Combine class is recommended for other to use.
First, initiate the class with one or many: paths to netcdf3, netcdf4, or hdf5 files; xr.Dataset objects (opened via xr.open_dataset); or h5py.File objects.
from hdf5tools import Combine
###############################
### Parameters
path1 = '/path/to/file1.nc'
path2 = '/path/to/file2.nc'
##############################
### Combine files
c1 = Combine([path1, path2])
If you want to do some kind of selection via the coordinates or only select some of the data variables/coordinates then use the .sel method (like in xarray). Be sure to read the docstrings for additional info about the input parameters.
c2 = c1.sel({'time': slice('2001-01-01', '2020-01-01'), 'latitude': slice(0, 10)}, include_data_vars=['precipitation'])
And finally, save the combined data to a single hdf5/netcdf4 file using the .to_hdf5 method. The only additional parameters that are important include the output which should be a path or a io.Bytes object, and the compression parameter. If you plan on using this file outside of the python environment, use gzip for compression, otherwise use lzf. The docstrings have more details.
output = '/path/to/output.nc'
c2.to_hdf5(output, compression='gzip')
If you’ve passed xr.Dataset objects to Combine, it will be slower than passing the file as a path on disk. Only pass xr.Dataset objects to Combine if you don’t want to write the intermediate file to disk before reading it into Combine.
The package also comes with a bonus function called xr_to_hdf5. It is a convenience function to convert a single xr.Dataset to an hdf5/netcdf4 file.
from hdf5tools import xr_to_hdf5
xr_to_hdf5(xr_dataset, output, compression='gzip')
Metadata
Release files for hdf5tools 0.5.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| hdf5tools-0.5.0.tar.gz | 29.6 kB | Without provenance | Details for hdf5tools-0.5.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| hdf5tools-0.5.0-py2.py3-none-any.whl | Python 2, Python 3 | none | any | Details for hdf5tools-0.5.0-py2.py3-none-any.whl |
Total release size: 63.1 kB
Release files / hdf5tools-0.5.0.tar.gz
| Download URL | hdf5tools-0.5.0.tar.gz |
|---|---|
| Size | 29.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
b1e733a91cec8631c4fea6eb58395235414ae92c945a0e149b3f115f75375b91
|
|
BLAKE2b-256 checksum How to use checksums |
854ed381806010233df674f2681aeb75ad5fe0e4f6437491f2dd3770cfa92c95
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.8.7
|
Release files / hdf5tools-0.5.0-py2.py3-none-any.whl
| Download URL | hdf5tools-0.5.0-py2.py3-none-any.whl |
|---|---|
| Size | 33.4 kB |
| Tags | Python 2 Python 3 |
|
SHA-256 checksum How to use checksums |
f90774ae23ab80956d23edda4c9bb23edf5739dbea0f27910bb5ee51dbea89c8
|
|
BLAKE2b-256 checksum How to use checksums |
8fdc2f8835b42df8aba7462a0c2d2cdb76db6b51b4902f119411adaa65a53a6e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.8.7
|
Security for hdf5tools 0.5.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.