node-graph-engine 0.1.0

Create node-based workflow

NodeGraph Engine

A platform for designing node-based workflows.

Supported engines

NodeGraph Engine ships with adaptors for a range of orchestration backends. Pick the engine that best matches your deployment target using the summary below and consult the documentation links for integration details.

Engine	Description
Direct	Run graphs locally inside the current Python process while capturing provenance.
Airflow	Materialise graphs as Airflow DAGs that can be scheduled and monitored in Apache Airflow.
Dagster	Launch graphs as Dagster jobs and surface executions in the Dagster UI via a configured instance.
Dask	Execute node jobs through Dask's threaded scheduler, keeping provenance in the local AiiDA profile while resolving dependencies and nested graphs automatically.
Prefect	Execute graphs as Prefect flows while streaming provenance back to the recorder.
Parsl	Dispatch tasks to Parsl executors for parallel and distributed execution without losing provenance fidelity.
Redun	Integrate with Redun workflows while preserving provenance throughout execution.
Jobflow	Bridge to the Jobflow workflow system for materials science workloads with consistent provenance records.
Executorlib (Pyiron)	Coordinate tasks with executorlib executors for provenance-rich high-throughput simulations.

Check the engine reference for a detailed overview of each adaptor and links to upstream documentation.

    pip install --upgrade --user node_graph_engine

Documentation

Check the docs and learn about the features.

Examples

A simple math calculation

from node_graph import NodeGraph, node

@node()
def add(x, y):
    return x + y

@node()
def multiply(x, y):
    return x + y

@node.graph()
def AddMultiply(x, y, z):
    the_sum = add(x=x, y=y).result
    return multiply(x=the_sum, y=z).result

Engines and provenance

Run graphs directly in Python:

from node_graph_engine.direct import DirectEngine

graph = AddMultiply.build(x=1, y=2, z=3)

engine = DirectEngine()
results = engine.run(graph)

The generated provenance graph:

License

MIT