Skip to main content

Create virtual Zarr stores from archival data using xarray API

Project description

VirtualiZarr

VirtualiZarr creates virtual Zarr stores for cloud-friendly access to archival data, using familiar xarray syntax.

VirtualiZarr (pronounced like "virtualize" but more piratey) grew out of discussions on the kerchunk repository, and is an attempt to provide the game-changing power of kerchunk in a zarr-native way, and with a familiar array-like API.

Please see the documentation

Development Status and Roadmap

VirtualiZarr version 1 (mostly) achieves feature parity with kerchunk's logic for combining datasets, providing an easier way to manipulate kerchunk references in memory and generate kerchunk reference files on disk.

Future VirtualiZarr development will focus on generalizing and upstreaming useful concepts into the Zarr specification, the Zarr-Python library, Xarray, and possibly some new packages.

We have a lot of ideas, including:

If you see other opportunities then we would love to hear your ideas!

Credits

This package was originally developed by Tom Nicholas whilst working at [C]Worthy, who deserve credit for allowing him to prioritise a generalizable open-source solution to the dataset virtualization problem. VirtualiZarr is now a community-owned multi-stakeholder project.

Licence

Apache 2.0

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

virtualizarr-1.0.0.tar.gz (61.6 kB view hashes)

Uploaded Source

Built Distribution

virtualizarr-1.0.0-py3-none-any.whl (49.2 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page