earthkit-workflows-anemoi

[!IMPORTANT] This software is Sandbox and subject to ECMWF's guidelines on Software Maturity.

Earthkit-Workflows-Anemoi is a Python library for connecting anemoi-inference to earthkit-workflows. Allowing for the inference tasks to be run as part of a larger DAG. It provides an API to directly create a graph consisting of initial condition retrieval and model execution, or to run inference off other source nodes which themselves are the initial conditions.

Installation

The package has a split dependency structure to allow flexible installation:

For workflow creation (minimal)

To create workflows without running them locally:

pip install earthkit-workflows-anemoi

This installs only the core dependencies needed to define and serialize workflows.

For workflow execution (full runtime)

To both create and execute workflows locally:

pip install 'earthkit-workflows-anemoi[runtime-inference]'

This includes anemoi-inference, required for local execution.

For development

git clone https://github.com/ecmwf/earthkit-workflows-anemoi.git
cd earthkit-workflows-anemoi
pip install -e '.[dev]'

Additionally you may want to install pre-commit hooks:

pip install pre-commit
pre-commit install

Quick Start

To create a graph relying on anemoi-inference to get the initial conditions the following can be used:

from earthkit.workflows.plugins import anemoi as anemoi_workflows

CKPT = {'huggingface': 'ecmwf/aifs-single-1.0'}

model_action = anemoi_workflows.fluent.from_input(CKPT, 'mars', '2022-01-01T00:00', lead_time = '7D', ensemble_members=range(1, 52))
model_action

Note: For models with multiple datasets (supported since anemoi-inference 0.11.0), the resulting action includes a dataset dimension. Use .select({"dataset": "era5"}) to select a specific dataset if needed.

Given other nodes as the initial conditions:

from earthkit.workflows.plugins import anemoi as anemoi_workflows
from earthkit.workflows import fluent

SOURCE_NODES: fluent.Action
CKPT = {'huggingface': 'ecmwf/aifs-single-1.0'}

SOURCE_NODES.anemoi.infer(CKPT, lead_time = '7D', ensemble_members = range(1, 52))