xpublish 0.3.0

Publish Xarray Datasets via a REST API.

Publish Xarray Datasets via a REST API.

Serverside: Publish a Xarray Dataset through a rest API

ds.rest.serve(host="0.0.0.0", port=9000)

Client-side: Connect to a published dataset

The published dataset can be accessed from various kinds of client applications. Here is an example of directly accessing the data from within Python:

import xarray as xr
import zarr
from fsspec.implementations.http import HTTPFileSystem

fs = HTTPFileSystem()
http_map = fs.get_mapper("http://0.0.0.0:9000")

# open as a zarr group
zg = zarr.open_consolidated(http_map, mode="r")

# or open as another Xarray Dataset
ds = xr.open_zarr(http_map, consolidated=True)

Why?

Xpublish lets you serve/share/publish Xarray Datasets via a web application.

The data and/or metadata in the Xarray Datasets can be exposed in various forms through pluggable REST API endpoints. Efficient, on-demand delivery of large datasets may be enabled with Dask on the server-side.

We are exploring applications of Xpublish that include:

• publish on-demand or derived data products

• turning xarray objects into streaming services (e.g. OPeNDAP)

How?

Under the hood, Xpublish is using a web app (FastAPI) that is exposing a REST-like API with builtin and/or user-defined endpoints.

For example, Xpublish provides by default a minimal Zarr compatible REST-like API with the following endpoints:

• .zmetadata: returns Zarr-formatted metadata keys as json strings.

• var/0.0.0: returns a variable data chunk as a binary string.