Python implementations of many OpenEO processes, dask-friendly by default.
Project description
OpenEO Processes Dask
openeo-processes-dask
is a collection of Python implementations of OpenEO processes based on the xarray/dask ecosystem. It is intended to be used alongside with openeo-pg-parser-networkx, which handles the parsing and execution of OpenEO process graphs. There you'll also find a tutorial on how to register process implementations from an arbitrary source (e.g. this repo) to the registry of available processes.
Installation
Install this project via pip:
pip install openeo-processes-dask
Note that by default this only installs the json process specs. In order to install the actual implementations, add the implementations
extra:
pip install openeo-processes-dask[implementations]
A subset of process implementations with heavy or unstable dependencies are hidden behind these further build variants:
openeo_processes_dask/process_implementations/ml
:pip install openeo-processes-dask[ml]
openeo_processes_dask/process_implementations/experimental
:pip install openeo-processes-dask[experimental]
Development environment
openeo-processes-dask requires poetry >1.2, see their docs for installation instructions.
Clone the repository with --recurse-submodules
to also fetch the process specs:
git clone --recurse-submodules git@github.com:Open-EO/openeo-processes-dask.git
To setup the python venv and install this project into it run:
poetry install --all-extras
To add a new core dependency run:
poetry add some_new_dependency
To add a new development dependency run:
poetry add some_new_dependency --group dev
To run the test suite run:
poetry run python -m pytest
Note that you can also use the virtual environment that's generated by poetry as the kernel for the ipynb notebooks.
Pre-commit hooks
This repo makes use of pre-commit hooks to enforce linting & a few sanity checks. In a fresh development setup, install the hooks using poetry run pre-commit install
. These will then automatically be checked against your changes before making the commit.
Specs
The json specs for the individual processes are tracked as a git submodule in openeo_processes_dask/specs/openeo-processes
.
The raw json for a specific process can be imported using from openeo_processes_dask.specs import reduce_dimension
.
To bump these specs to a later version use:
git -C openeo_processes_dask/specs/openeo-processes checkout <tag>
git add openeo_processes_dask/specs/openeo-processes
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for openeo_processes_dask-2024.5.3.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | bb0aee60ce86e4ae57000b9db8bb2ab0ed9c218e33dfe6cb6c273496fc54c965 |
|
MD5 | 194dbe17a75e2eac00f8773c1669fe47 |
|
BLAKE2b-256 | a9dc340a937b281349aff68d237ae52d2b30145e60ea43da49d325d6e4a3a7a5 |
Hashes for openeo_processes_dask-2024.5.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 63d7a3aa08ca5436e50b45f73ec1fb7628661de1e16102992a7c62656f42a9a1 |
|
MD5 | 788d34f3e5e194506c880846d8041f1d |
|
BLAKE2b-256 | 08ce7238c5c1e47d1461a02351e29a3c007e410afd31acceb13df378bdd2e619 |