Skip to main content

Xarray accessors for publishing datasets as web applications

Project description

GitHub Workflow Status Documentation Status Binder

xpublish

Publish Xarray Datasets via a Zarr compatible REST API

Server side: Publish a xarray dataset as a rest API

ds.rest.serve(host="0.0.0.0", port=9000)

Client side: Connect to a published dataset

import xarray as xr
import zarr
from fsspec.implementations.http import HTTPFileSystem

fs = HTTPFileSystem()
http_map = fs.get_mapper('http://0.0.0.0:9000')

# open as a zarr group
zg = zarr.open_consolidated(http_map, mode='r')

# open as another xarray dataset
xr.open_zarr(http_map, consolidated=True)

Why?

xpublish lets you serve/share/publish xarray datasets via a web application. The data in the xarray datasets (on the server side) can be backed by dask to facilitate on-demand computation via a simple REST API. We are exploring applications of xpublish that include:

  • publish on-demand derived data products
  • turning xarray objects into streaming services (e.g. OPENDAP)

How?

Under the hood, xpublish is using a web app (FastAPI) that is exposing a minimal Zarr compatible REST-like API. Key attributes of the API are:

  • serves a Zarr store API from the root of the dataset
  • provides Zarr metadata keys (.zmetadata) as json strings.
  • provides access to data keys (e.g. var/0.0.0) as binary strings.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for xpublish, version 0.0.2
Filename, size File type Python version Upload date Hashes
Filename, size xpublish-0.0.2-py3-none-any.whl (6.4 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size xpublish-0.0.2.tar.gz (22.5 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page