Skip to main content

Interface for using cubed with xarray for parallel computation.

Project description

Note: this is a proof-of-concept, and many things are incomplete, untested, or don't work.

cubed-xarray

Interface for using cubed with xarray.

Requirements

  • Cubed version >=0.14.2
  • Xarray version >=2024.02.0

Installation

Install via pip

pip install cubed-xarray

or conda

conda install -c conda-forge cubed-xarray

Importing

You don't need to import this package in user code. Once poperly installed, xarray should automatically become aware of this package via the magic of entrypoints.

Usage

Xarray objects backed by cubed arrays can be created either by:

  1. Passing existing cubed.Array objects to the data argument of xarray constructors,
  2. Calling .chunk on xarray objects,
  3. Passing a chunks argument to xarray.open_dataset.

In (2) and (3) the choice to use cubed.Array instead of dask.array.Array is made by passing the keyword argument chunked_array_type='cubed'. To pass arguments to the constructor of cubed.Array you should pass them via the dictionary from_array_kwargs, e.g. from_array_kwargs={'spec': cubed.Spec(allowed_mem='2GB')}.

If cubed and cubed-xarray are installed but dask is not, then specifying chunked_array_type is not necessary, as the entrypoints system will then default to the only chunked parallel backend available (i.e. cubed).

Sharp Edges 🔪

Some things almost certainly won't work yet:

and some other things might work but have not yet been tried:

  • Saving to formats other than zarr

In general a bug could take the form of an error, or of a silent attempt to coerce the array type to numpy by immediately computing the underlying array.

Tests

Integration tests for wrapping cubed with xarray also live in this repository.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

cubed-xarray-0.0.7.tar.gz (19.9 kB view details)

Uploaded Source

Built Distribution

cubed_xarray-0.0.7-py3-none-any.whl (16.5 kB view details)

Uploaded Python 3

File details

Details for the file cubed-xarray-0.0.7.tar.gz.

File metadata

  • Download URL: cubed-xarray-0.0.7.tar.gz
  • Upload date:
  • Size: 19.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.9.16

File hashes

Hashes for cubed-xarray-0.0.7.tar.gz
Algorithm Hash digest
SHA256 b08a969656c9c05c7168f9eee78911b73d6ab31781c317984b490c8e9d984eac
MD5 b938161ff5abb78ac9f08c29d50835a4
BLAKE2b-256 013cb5dd75d70fb05e50771148296c20d415e7f20e1c9d86e90e106677ea4388

See more details on using hashes here.

File details

Details for the file cubed_xarray-0.0.7-py3-none-any.whl.

File metadata

  • Download URL: cubed_xarray-0.0.7-py3-none-any.whl
  • Upload date:
  • Size: 16.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.9.16

File hashes

Hashes for cubed_xarray-0.0.7-py3-none-any.whl
Algorithm Hash digest
SHA256 83aad6439c4ab5b58601f2ef806e45aac64adc43a84ddec21c5507ec13c8af35
MD5 f26a68202947c28bdd3b39d72d6057a5
BLAKE2b-256 e7bfd3e75275497bc0a84f3420c15952dd63beb0b2b6cb9fedbad32a640f0750

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page