h5netcdf 0.2.1

netCDF4 via h5py

A Python interface for the netCDF4 file-format that reads and writes HDF5 files API directly via h5py, without relying on the Unidata netCDF library.

This is an experimental project. It currently passes basic tests for reading and writing netCDF4 files with Python, but it has not been tested for compatibility with other netCDF4 interfaces.

Motivations

Why did I write h5netcdf? Well, here are a few reasons:

• To prove it could be done (it seemed like an obvious thing to do) and that netCDF4 is not actually that complicated.

• We’ve seen occasional reports of better performance with h5py than netCDF4-python that I wanted to be able to verify. For some workflows, h5netcdf has been reported to be almost 4x faster than netCDF4-python.

• h5py seems to have thought through multi-threading pretty carefully, so this in particular seems like a case where things could make a difference. I’ve started to care about this because I recently hooked up a multi-threaded backend to xray.

• It’s one less massive binary dependency (netCDF C). Anecdotally, HDF5 users seem to be unexcited about switching to netCDF – hopefully this will convince them that they are really the same thing!

• Finally, side-stepping the netCDF C library (and Cython bindings to it) gives us an easier way to identify the source of performance issues and bugs.

Install

Ensure you have h5py installed (I recommend using conda). Then: pip install h5netcdf

Usage

h5netcdf has two APIs, a new API and a legacy API.

New API

The new API supports direct hierarchical access of variables and groups. Its design is an adaptation of h5py to the netCDF data model. For example:

import h5netcdf
import numpy as np

with h5netcdf.File('mydata.nc', 'w') as f:
    # set dimensions with a dictionary
    f.dimensions = {'x': 5}
    # and update them with a dict-like interface
    # f.dimensions['x'] = 5
    # f.dimensions.update({'x': 5})

    v = f.create_variable('hello', ('x',), float)
    v[:] = np.ones(5)

    # you don't need to create groups first
    # you also don't need to create dimensions first if you supply data
    # with the new variable
    v = f.create_variable('/grouped/data', ('y',), data=np.arange(10))

    # access and modify attributes with a dict-like interface
    v.attrs['foo'] = 'bar'

    # you can access variables and groups directly using a hierarchical
    # keys like h5py
    print(f['/grouped/data'])

Warning: The design of the new API is *not yet finished*. I only recommended using it for experiments. Please share your feedback in this GitHub issue.

Legacy API

The legacy API is designed for compatibility with netCDF4-python. To use it, import h5netcdf.legacyapi:

import h5netcdf.legacyapi as netCDF4
# everything here would also work with this instead:
# import netCDF4
import numpy as np

with netCDF4.Dataset('mydata.nc', 'w') as ds:
    ds.createDimension('x', 5)
    v = ds.createVariable('hello', float, ('x',))
    v[:] = np.ones(5)

    g = ds.createGroup('grouped')
    g.createDimension('y', 10)
    g.createVariable('data', 'i8', ('y',))
    v = g['data']
    v[:] = np.arange(10)
    v.foo = 'bar'
    print(ds.groups['grouped'].variables['data'])

License

3-clause BSD