Parse OpenEO process graphs from JSON to traversible Python objects.
Project description
OpenEO Process Graph Parser (Python & networkx)
Python package to parse OpenEO process graphs from raw JSON into fully traversible networkx
graph objects.
This package is an evolution of the openeo-pg-parser-python package.
Installation
This package can be installed with pip:
pip install openeo-pg-parser-networkx
To enable plotting also install the plot
extra:
pip install openeo-pg-parser-networkx[plot]
Currently Python versions 3.9 and 3.10 are supported.
Basic usage
(An example notebook of using openeo-pg-parser-networkx
together with a process implementation source like openeo-processes-dask
can be found in openeo-pg-parser-networkx/examples/01_minibackend_demo.ipynb
.)
Parse a JSON OpenEO process graph:
from openeo_pg_parser_networkx import OpenEOProcessGraph
EVI_GRAPH_PATH = "../tests/data/graphs/pg-evi-example.json"
parsed_graph = OpenEOProcessGraph.from_file(EVI_GRAPH_PATH)
> Deserialised process graph into nested structure
> Walking node root-7ecd43ed-b694-4a18-8805-eb366d277c8e
> Walking node mintime-7ecd43ed-b694-4a18-8805-eb366d277c8e
> Walking node min-80d5faba-c298-4d2f-82f5-be06ee417565
> Walking node evi-7ecd43ed-b694-4a18-8805-eb366d277c8e
> Walking node m3-657ee106-6571-4509-a1cf-59f212286011
> Walking node div-657ee106-6571-4509-a1cf-59f212286011
> Walking node sub-657ee106-6571-4509-a1cf-59f212286011
> Walking node nir-657ee106-6571-4509-a1cf-59f212286011
> Walking node red-657ee106-6571-4509-a1cf-59f212286011
> Walking node sum-657ee106-6571-4509-a1cf-59f212286011
> Walking node nir-657ee106-6571-4509-a1cf-59f212286011
> Walking node m1-657ee106-6571-4509-a1cf-59f212286011
> Walking node red-657ee106-6571-4509-a1cf-59f212286011
> Walking node m2-657ee106-6571-4509-a1cf-59f212286011
> Walking node blue-657ee106-6571-4509-a1cf-59f212286011
> Walking node load_collection-7ecd43ed-b694-4a18-8805-eb366d277c8e
Plot it:
parsed_graph.plot()
To execute a process graph, OpenEOProcessGraph
needs to know which Python code to call for each of the nodes in the graph. This information is provided by a "process registry", which is basically a dictionary that maps each process_id
to its actual Python implementation as a Callable
.
Register process implementations to a "process registry":
The ProcessRegistry
object also allows registering wrapper functions that will be wrapped around each registered process implementation.
See openeo-processes-dask for an example of a wrapper function that resolves incoming parameters.
from openeo_pg_parser_networkx import ProcessRegistry
from openeo_processes_dask.process_implementations import apply, ndvi, multiply, load_collection, save_result
from openeo_processes_dask.core import process
# `process` is wrapped around each registered implementation
process_registry = ProcessRegistry(wrap_funcs=[process])
process_registry["apply"] = apply
process_registry["ndvi"] = ndvi
process_registry["multiply"] = multiply
process_registry["load_collection"] = load_collection
process_registry["save_result"] = save_result
Build an executable callable from the process graph:
pg_callable = parsed_graph.to_callable(process_registry=process_registry)
Execute that callable like a normal Python function:
pg_callable
> Running process load_collection
> Running process apply
> ...
Development environment
openeo-pg-parser-networkx
requires poetry >1.2
, see their docs for installation instructions.
To setup the python venv and install this project into it run:
poetry install
To add a new core dependency run:
poetry add some_new_dependency
To add a new development dependency run:
poetry add some_new_dependency --group dev
To run the test suite run:
poetry run python -m pytest
Note that you can also use the virtual environment that's generated by poetry as the kernel for the ipynb notebooks.
Pre-commit hooks
This repo makes use of pre-commit hooks to enforce linting & a few sanity checks.
In a fresh development setup, install the hooks using poetry run pre-commit install
.
These will then automatically be checked against your changes before making the commit.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for openeo_pg_parser_networkx-2023.3.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 58d63238f80394e632c8d6b79fc0ebe73421b00230685bc8f0fde3ea030fbc2d |
|
MD5 | 2ef3983257bb014452a1adf28b9273eb |
|
BLAKE2b-256 | 7f4e50cd844f176021e9e7dbbcf077762d702a7e41629fc0b78b20a8074a5c7b |
Hashes for openeo_pg_parser_networkx-2023.3.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 77e6af1ca1232559a8b983e68895d6c9a13c6e59b613e64886631bd9696b44dd |
|
MD5 | 6d53a919b4dbd986d6ac79b6d02ee732 |
|
BLAKE2b-256 | ba587489810e276667782f6f968bf81100a607ba4878581792ec949294152b2e |