openeo-processes-dask 2024.11.0

Python implementations of many OpenEO processes, dask-friendly by default.

OpenEO Processes Dask

openeo-processes-dask is a collection of Python implementations of OpenEO processes based on the xarray/dask ecosystem. It is intended to be used alongside with openeo-pg-parser-networkx, which handles the parsing and execution of OpenEO process graphs. There you'll also find a tutorial on how to register process implementations from an arbitrary source (e.g. this repo) to the registry of available processes.

Installation

Install this project via pip:

pip install openeo-processes-dask

Note that by default this only installs the json process specs. In order to install the actual implementations, add the implementations extra:

pip install openeo-processes-dask[implementations]

A subset of process implementations with heavy or unstable dependencies are hidden behind these further build variants:

• openeo_processes_dask/process_implementations/ml: pip install openeo-processes-dask[ml]
• openeo_processes_dask/process_implementations/experimental: pip install openeo-processes-dask[experimental]

Development environment

openeo-processes-dask requires poetry >1.2, see their docs for installation instructions.

Clone the repository with --recurse-submodules to also fetch the process specs:

git clone --recurse-submodules [e-mail]:Open-EO/openeo-processes-dask.git

To setup the python venv and install this project into it run:

poetry install --all-extras

To add a new core dependency run:

poetry add some_new_dependency

To add a new development dependency run:

poetry add some_new_dependency --group dev

To run the test suite run:

poetry run python -m pytest

Note that you can also use the virtual environment that's generated by poetry as the kernel for the ipynb notebooks.

Pre-commit hooks

This repo makes use of pre-commit hooks to enforce linting & a few sanity checks. In a fresh development setup, install the hooks using poetry run pre-commit install. These will then automatically be checked against your changes before making the commit.

Specs

The json specs for the individual processes are tracked as a git submodule in openeo_processes_dask/specs/openeo-processes. The raw json for a specific process can be imported using from openeo_processes_dask.specs import reduce_dimension.

To bump these specs to a later version use: git -C openeo_processes_dask/specs/openeo-processes checkout <tag> git add openeo_processes_dask/specs/openeo-processes