foampilot-csc 1.0.3

Python orchestration for OpenFOAM, SmartSim, and machine-learning workflows on CSC systems

FoamPilot CSC

FoamPilot is a typed Python interface for coupling OpenFOAM simulations with Python, JAX, ONNX, SmartSim, and SmartRedis workflows.

It does not replace OpenFOAM. OpenFOAM remains responsible for the mesh, discretisation, boundary conditions, pressure–velocity coupling, linear solvers, and MPI domain decomposition. FoamPilot provides the layer that describes which OpenFOAM fields should be exchanged, how they map to external operators, when the exchange occurs, and how the resulting fields are returned to the running simulation.

┌──────────────────────────┐
│        OpenFOAM          │
│                          │
│ mesh and field registry  │
│ finite-volume operators  │
│ PDE assembly and solve   │
│ MPI decomposition        │
└─────────────┬────────────┘
              │ SmartRedis tensors
┌─────────────▼────────────┐
│        FoamPilot         │
│                          │
│ case inspection          │
│ field specifications     │
│ expression validation    │
│ operator contracts       │
│ timestep synchronisation │
│ MPI-aware data exchange  │
└─────────────┬────────────┘
              │
┌─────────────▼────────────┐
│   External computation   │
│                          │
│ Python / NumPy / JAX     │
│ ONNX / machine learning  │
│ custom closure models    │
└──────────────────────────┘

The main use case is online coupling:

OpenFOAM field
    ↓
SmartRedis
    ↓
FoamPilot operator
    ↓
Python, JAX, or ONNX calculation
    ↓
SmartRedis
    ↓
updated OpenFOAM field

Current development is focused on external turbulence and reacting-flow closures, with infrastructure for field streaming, round-trip field exchange, external Smagorinsky modelling, and external k-equation modelling.

---

Package layout

components/foampilot/
├── pyproject.toml
├── README.md
└── foampilot/
    ├── __init__.py
    ├── case.py
    ├── environment.py
    ├── feature.py
    ├── field.py
    ├── run.py
    ├── math/
    ├── operator/
    ├── coupling/
    ├── closure/
    └── contracts/
        ├── closure_models.yaml
        ├── openfoam_operators.yaml
        └── math_examples.yaml

The distribution and import names are different:

pip install foampilot-csc

import foampilot as fp

FoamPilot is maintained together with the SmartSim-CSC OpenFOAM integration. The Python package and compiled OpenFOAM libraries should be taken from compatible SmartSim-CSC revisions.

---

1. Responsibilities

OpenFOAM

OpenFOAM remains responsible for:

• mesh topology and geometry
• field registration
• finite-volume operators such as grad, div, and laplacian
• boundary conditions
• equation assembly
• linear-system solution
• pressure–velocity coupling
• turbulence transport equations
• MPI decomposition and parallel execution

SmartSim

SmartSim remains responsible for:

• launching the SmartRedis database
• launching OpenFOAM and other applications
• local or Slurm execution
• resource and batch configuration
• process monitoring
• experiment summaries

SmartRedis

SmartRedis provides:

• tensor exchange
• datasets and metadata
• model registration
• backend model execution
• communication between OpenFOAM and Python

FoamPilot

FoamPilot provides:

• OpenFOAM case inspection
• mesh and solver validation
• typed OpenFOAM field definitions
• validated field expressions
• Python and ONNX operator contracts
• input and output relay configuration
• timestep synchronisation
• MPI-aware tensor reconstruction and partitioning
• external closure configuration
• runtime step objects with physical time and field access

FoamPilot intentionally does not hide SmartSim resource configuration. The OpenFOAM coupling API and the HPC launch API remain separate.

---

2. Core concepts

A complete FoamPilot workflow is built from five main concepts:

FoamCase
    │
    ├── FieldSpec
    ├── Operator
    ├── FoamFieldRelay
    └── FoamCoupling
            │
            └── FoamCouplingStep

FoamCase

Represents one OpenFOAM case and provides case inspection, validation, configuration, and execution metadata.

FieldSpec

Maps an external operator argument to an OpenFOAM field or field expression.

Operator

Defines the external computation: its callable or backend, inputs, outputs, and constant keyword arguments.

FoamFieldRelay

Defines which fields move in which direction and at what transfer interval.

FoamCoupling

Combines input relays, an optional operator, and output relays into one synchronised workflow.

FoamCouplingStep

Represents one OpenFOAM exchange point. It contains the current time index, physical time, input fields, and methods for sending outputs.

---

3. Creating and preparing a case

from pathlib import Path

import foampilot as fp

case = fp.FoamCase(
    path=Path("/path/to/openfoam/case"),
    simulation_type="les",
    poll_interval=0.005,
    poll_timeout=60.0,
)

The main arguments are:

Argument	Meaning
path	OpenFOAM case directory
simulation_type	Intended simulation category, such as laminar or les
poll_interval	SmartRedis polling interval in seconds
poll_timeout	Maximum wait time for a required exchange

Prepare and validate the case with:

report = case.initialize(
    clean=True,
    block_mesh=True,
    validate_mesh=True,
    validate_solver=True,
    n_subdomains=8,
)

This can:

• clean previous generated results
• run blockMesh
• run mesh validation
• validate the configured solver
• prepare decomposition for MPI execution
• inspect available OpenFOAM fields and objects

The returned report exposes useful information:

print(report.available_objects)
print(report.available_fields)

The resolved OpenFOAM executable and arguments are available through:

print(case.execution.exe)
print(case.execution.exe_args)

These values can be passed directly to SmartSim:

run_settings = experiment.create_run_settings(
    exe=case.execution.exe,
    exe_args=case.execution.exe_args,
)

---

4. OpenFOAM fields and expressions

FoamPilot distinguishes between primitive fields and derived expressions.

Primitive fields already exist in the OpenFOAM object registry:

U
p
k
nut
rho
T

Derived expressions are evaluated from current OpenFOAM fields:

grad(U)
div(U)
curl(U)
laplacian(k)
symm(grad(U))
dev(symm(grad(U)))

Mesh-dependent differential operators are evaluated by OpenFOAM, not by NumPy. This preserves OpenFOAM geometry, discretisation schemes, boundary conditions, non-orthogonal corrections, and parallel behaviour.

---

5. fp.Math

fp.Math provides validated OpenFOAM expression builders.

fp.Math.grad("U")
fp.Math.div("phi", "U")
fp.Math.curl("U")
fp.Math.laplacian("nu", "U")
fp.Math.mag("U")
fp.Math.symm(fp.Math.grad("U"))
fp.Math.dev(fp.Math.symm(fp.Math.grad("U")))

For example:

strain_expression = fp.Math.symm(
    fp.Math.grad("U")
)

deviatoric_expression = fp.Math.dev(
    strain_expression
)

produces:

symm(grad(U))
dev(symm(grad(U)))

The expression parser validates operator names, argument counts, nested expressions, and primitive field names before the case is executed.

Array dispatch

Some fp.Math operations also support NumPy or JAX arrays.

strain_rate = fp.Math.symm(velocity_grad)
strain_rate_magnitude = fp.Math.mag(strain_rate)

In this form, fp.Math performs the corresponding algebraic array operation.

The intended division of responsibility is:

mesh-dependent differential operation
    → OpenFOAM expression

array algebra after field transfer
    → NumPy, JAX, or fp.Math array operation

Discovering supported operations

fp.Math.describe()

Filter by operator:

fp.Math.describe(
    operator="laplacian",
)

Filter by one or more contexts:

fp.Math.describe(
    context=[
        "reacting",
        "compressible",
    ],
)

The description system reads the operator and example contracts distributed with FoamPilot.

---

6. FieldSpec

A FieldSpec connects a Python argument or output name to an OpenFOAM field.

velocity_gradient = fp.FieldSpec(
    "grad(U)",
    "tensor",
)

A typical input mapping is:

inputs={
    "velocity_grad": fp.FieldSpec(
        "grad(U)",
        "tensor",
    ),
}

This means:

OpenFOAM expression: grad(U)
Python argument:     velocity_grad
Physical type:       tensor

The external function does not need to use OpenFOAM field names:

def model(velocity_grad):
    ...

Representative field types are:

scalar
vector
tensor

FoamPilot uses this type information to validate and reshape transferred arrays. For example, a tensor transferred as nine components per cell can be presented to the Python operator as:

(n_cells, 3, 3)

and converted back to the OpenFOAM transfer layout when returned.

---

7. Operators

An operator is a typed contract for an external computation.

It defines:

• the backend or callable
• named Python inputs
• OpenFOAM input fields
• named Python outputs
• OpenFOAM output fields
• field types
• constant keyword arguments

Python operator

import numpy as np

import foampilot as fp


def smagorinsky_function(
    velocity_grad,
    filter_width,
    C_s=0.17,
):
    """Compute Smagorinsky eddy viscosity."""

    strain_rate = fp.Math.symm(
        velocity_grad
    )

    strain_rate_magnitude = (
        np.sqrt(2.0)
        * fp.Math.mag(strain_rate)
    )

    eddy_viscosity = (
        C_s
        * filter_width
    )**2 * strain_rate_magnitude

    return eddy_viscosity

Register the function as an operator:

smagorinsky_operator = fp.Operator.python(
    function=smagorinsky_function,
    inputs={
        "velocity_grad": fp.FieldSpec(
            "grad(U)",
            "tensor",
        ),
        "filter_width": fp.FieldSpec(
            "delta",
            "scalar",
        ),
    },
    outputs={
        "eddy_viscosity": fp.FieldSpec(
            "nut",
            "scalar",
        ),
    },
    kwargs={
        "C_s": 0.17,
    },
)

The contract can be read as:

grad(U) → velocity_grad
delta   → filter_width

smagorinsky_function(...)

eddy_viscosity → nut

The function may use standard Python, NumPy, JAX, or an arbitrary user model.

ONNX operator

FoamPilot also provides an ONNX operator interface.

onnx_operator = fp.Operator.onnx(
    ...,
)

Python and ONNX operators share the same higher-level workflow:

typed inputs
    ↓
backend execution
    ↓
typed outputs

---

8. Configuring field exchange

Use case.configure() to define the direction and timing of data transfer.

OpenFOAM to SmartRedis

inputs = case.configure(
    fields=[
        "U",
        fp.Math.grad("U"),
    ],
    direction="of -> db",
    transfer_at="each",
)

SmartRedis to OpenFOAM

outputs = case.configure(
    fields=["U"],
    direction="db -> of",
    transfer_at="each",
)

Supported direction names are normalised internally:

of -> db
db -> of

Representative transfer values are:

each
write
integer stride

Online closures normally use:

transfer_at="each"

because the returned closure value is required during the simulation.

Configure directly from an operator

closure_inputs = case.configure(
    operator=smagorinsky_operator,
    direction="of -> db",
    transfer_at="each",
)

closure_outputs = case.configure(
    operator=smagorinsky_operator,
    direction="db -> of",
    transfer_at="each",
)

This avoids manually duplicating the operator input and output field names.

---

9. Coupling

Combine relays and an optional operator with fp.couple().

les_coupling = fp.couple(
    inputs=closure_inputs,
    outputs=closure_outputs,
    operator=smagorinsky_operator,
)

Conceptually:

Input relay             Operator             Output relay

grad(U) ───────┐
               ├── Smagorinsky model ───────→ nut
delta ─────────┘

The coupling validates that:

• required operator inputs are available
• required operator outputs have destinations
• relay directions are correct
• field names and types are compatible
• closure requirements are satisfied when a closure is active

A coupling without an operator is also useful for direct round-trip tests:

feature_exchange = fp.couple(
    inputs=feature_inputs,
    outputs=feature_outputs,
)

---

10. Timestep execution

Start the OpenFOAM model through SmartSim and iterate over exchange steps.

experiment.start(
    openfoam_model,
    block=False,
    summary=True,
)

for step in les_coupling.steps(
    client=client,
    experiment=experiment,
    model=openfoam_model,
):
    outputs = les_coupling.evaluate(step)
    step.send(**outputs)

The runtime sequence is:

OpenFOAM reaches an exchange point
    ↓
OpenFOAM publishes input fields
    ↓
FoamPilot waits for all required tensors
    ↓
FoamPilot reconstructs typed arrays
    ↓
external operator is evaluated
    ↓
outputs are validated
    ↓
FoamPilot partitions outputs by MPI rank
    ↓
outputs are written to SmartRedis
    ↓
OpenFOAM receives the outputs
    ↓
OpenFOAM continues

Accessing step data

print(step.time_index)
print(step.time)

velocity_gradient = step["grad(U)"]
filter_width = step["delta"]

time_index is the integer OpenFOAM step index.

time is the physical simulation time.

Evaluating and sending

outputs = les_coupling.evaluate(step)

step.send(
    **outputs
)

The coupling maps OpenFOAM fields to operator arguments, executes the operator, maps the result to OpenFOAM field names, and sends rank-partitioned tensors.

---

11. MPI behaviour

OpenFOAM MPI ranks own separate local cell partitions.

rank 0 → local cells
rank 1 → local cells
...
rank N → local cells

FoamPilot reconstructs the rank-local tensors into one external array:

(total_cells, ...)

When an output is returned, FoamPilot splits it back into the original rank-local partitions before sending it to OpenFOAM.

Python output
    ↓
rank 0 output partition
rank 1 output partition
...
rank N output partition

This MPI-aware reconstruction is a central part of the coupling layer.

---

12. External LES closures

FoamPilot currently provides OpenFOAM integration for external incompressible LES closures.

smartSimNut

case.configure_closure(
    model="smartSimNut",
)

Typical flow:

OpenFOAM:
    grad(U), delta
        ↓
Python:
    compute nut
        ↓
OpenFOAM:
    receive nut
    update turbulence viscosity
    use it in the next applicable momentum-equation assembly

This is suitable for algebraic SGS models such as Smagorinsky.

smartSimKEqn

Representative inputs:

k
grad(U)
delta

Representative outputs:

nut
kProduction
kDissipationCoeff

The intended split is:

external operator:
    closure algebra and coefficients

OpenFOAM:
    finite-volume k equation
    boundary conditions
    linear solve
    field correction

The exact field contract is stored in:

foampilot/contracts/closure_models.yaml

configure_closure() validates the selected model, case type, required fields, operator contract, and online transfer interval.

---

13. Complete external Smagorinsky workflow

import numpy as np
from smartredis import Client
from smartsim import Experiment

import foampilot as fp


def smagorinsky_function(
    velocity_grad,
    filter_width,
    C_s=0.17,
):
    strain_rate = fp.Math.symm(
        velocity_grad
    )

    strain_rate_magnitude = (
        np.sqrt(2.0)
        * fp.Math.mag(strain_rate)
    )

    eddy_viscosity = (
        C_s
        * filter_width
    )**2 * strain_rate_magnitude

    return eddy_viscosity


smagorinsky_operator = fp.Operator.python(
    function=smagorinsky_function,
    inputs={
        "velocity_grad": fp.FieldSpec(
            "grad(U)",
            "tensor",
        ),
        "filter_width": fp.FieldSpec(
            "delta",
            "scalar",
        ),
    },
    outputs={
        "eddy_viscosity": fp.FieldSpec(
            "nut",
            "scalar",
        ),
    },
    kwargs={
        "C_s": 0.17,
    },
)

case = fp.FoamCase(
    path="/path/to/planeChannel",
    simulation_type="les",
    poll_interval=0.005,
    poll_timeout=60.0,
)

case.initialize(
    clean=True,
    block_mesh=True,
    validate_mesh=True,
    validate_solver=True,
    n_subdomains=8,
)

case.configure_closure(
    model="smartSimNut",
)

closure_inputs = case.configure(
    operator=smagorinsky_operator,
    direction="of -> db",
    transfer_at="each",
)

closure_outputs = case.configure(
    operator=smagorinsky_operator,
    direction="db -> of",
    transfer_at="each",
)

les_coupling = fp.couple(
    inputs=closure_inputs,
    outputs=closure_outputs,
    operator=smagorinsky_operator,
)

The SmartSim database and OpenFOAM model remain explicit:

experiment = Experiment(
    name="openfoam-external-smagorinsky",
    launcher="slurm",
)

database = experiment.create_database(
    db_nodes=1,
    port=2026,
    interface="ib0",
    batch=True,
    time="00:15:00",
    account="project_xxxxxxx",
)

experiment.generate(
    database,
    overwrite=True,
)

experiment.start(
    database,
    block=False,
)

client = Client(
    address=database.get_address()[0],
    cluster=False,
)

openfoam_batch_settings = experiment.create_batch_settings(
    nodes=1,
    time="00:15:00",
    account="project_xxxxxxx",
)

openfoam_batch_settings.set_partition("small")
openfoam_batch_settings.batch_args["ntasks-per-node"] = "8"
openfoam_batch_settings.batch_args["mem"] = "16G"
openfoam_batch_settings.set_cpus_per_task(1)

run_settings = experiment.create_run_settings(
    exe=case.execution.exe,
    exe_args=case.execution.exe_args,
)

run_settings.set_tasks(8)
run_settings.set_tasks_per_node(8)
run_settings.set_cpus_per_task(1)

openfoam_model = experiment.create_model(
    name=case.name,
    run_settings=run_settings,
    batch_settings=openfoam_batch_settings,
)

experiment.generate(
    openfoam_model,
    overwrite=True,
)

experiment.start(
    openfoam_model,
    block=False,
    summary=True,
)

Run the online coupling loop:

for step in les_coupling.steps(
    client=client,
    experiment=experiment,
    model=openfoam_model,
):
    outputs = les_coupling.evaluate(step)
    step.send(**outputs)

    nut = outputs["nut"]

    print(
        f"time_index={step.time_index}, "
        f"physical_time={step.time:.6f}, "
        f"grad(U)={step['grad(U)'].shape}, "
        f"delta={step['delta'].shape}, "
        f"nut={nut.shape}, "
        f"nut min={nut.min():.6e}, "
        f"nut mean={nut.mean():.6e}, "
        f"nut max={nut.max():.6e}"
    )

status = fp.wait_for_model(
    experiment,
    openfoam_model,
)

print(f"OpenFOAM status: {status}")

experiment.stop(
    database
)

The Slurm account, partition, memory, port, interface, and task counts are site-specific and intentionally remain SmartSim configuration.

---

14. Environment loading

A complete runtime requires:

• SmartSim
• SmartRedis Python client
• native SmartRedis libraries
• the compiled OpenFOAM integration
• a compatible OpenFOAM runtime
• FoamPilot

On CSC installations, the environment loader is normally selected through:

import os

os.environ["SMARTSIM_ENV_LOADER"] = (
    "/path/to/Python4SmartSim.sh"
)

Typical runtime variables include:

WM_PROJECT_DIR
WM_PROJECT_VERSION
FOAM_USER_APPBIN
FOAM_USER_LIBBIN
SMARTREDIS_DIR
LD_LIBRARY_PATH

Use load_environment() when explicit loading or validation is required:

environment = fp.load_environment()

---

15. Process monitoring

Use wait_for_model() to wait until a SmartSim model reaches a terminal state.

status = fp.wait_for_model(
    experiment,
    openfoam_model,
)

A completed status is returned. Failed execution raises an error rather than silently returning incomplete coupling results.

---

16. Current public API

The main public API currently includes:

fp.FoamCase
fp.FoamCaseReport
fp.FoamField
fp.FoamBoundary

fp.FieldSpec
fp.Operator
fp.PythonOperator
fp.OnnxOperator

fp.Math

fp.FoamFieldRelay
fp.FoamCoupling
fp.FoamCouplingStep
fp.couple

fp.FoamClosure

fp.FoamExecution
fp.FoamRun
fp.FoamFrame

fp.load_environment
fp.wait_for_model

Important FoamCase members include:

case.initialize(...)
case.configure(...)
case.configure_closure(...)

case.execution
case.name
case.field_names
case.boundary_names

The API is under active development. Compatibility aliases and legacy interfaces are not guaranteed to remain available between development revisions.

---

17. Current capabilities

The current implementation supports:

• OpenFOAM case inspection
• solver and mesh validation
• OpenFOAM execution metadata
• primitive field exchange
• derived OpenFOAM field expressions
• nested expression validation
• grad
• div
• curl
• laplacian
• mag
• symm
• dev
• NumPy/JAX array dispatch for selected algebraic operations
• Python operator contracts
• ONNX operator contracts
• typed scalar, vector, and tensor fields
• OpenFOAM-to-database relays
• database-to-OpenFOAM relays
• timestep-index and physical-time tracking
• MPI-aware input reconstruction
• MPI-aware output partitioning
• direct round-trip field exchange
• external smartSimNut closure coupling
• external smartSimKEqn closure coupling
• SmartSim model status monitoring
• contract-driven operator descriptions and examples

---

18. Current boundaries

FoamPilot does not currently provide a universal runtime setter for every OpenFOAM parameter.

The following objects may require different update mechanisms:

volScalarField or volVectorField
dictionary entry
dimensioned scalar
transport-model property
thermodynamic derived property
cached model state

A field that exists in the OpenFOAM registry can often be relayed generically. A value read from transportProperties, or a value recomputed internally by a thermodynamic model, may require model-specific runtime handling.

FoamPilot also does not attempt to reproduce mesh-dependent finite-volume operators in NumPy. Such operations remain in OpenFOAM.

The current design is strongest for workflows of the form:

OpenFOAM:
    compute mesh-dependent features

external model:
    compute algebraic or machine-learning closure values

OpenFOAM:
    solve the governing equations

---

19. Validation priorities

Before production use, each new closure should verify:

1. field names and physical types
2. tensor shapes
3. dimensions and units
4. MPI reconstruction and partitioning
5. transfer timing
6. update frequency
7. boundary-condition behaviour
8. numerical agreement with an OpenFOAM reference
9. runtime overhead
10. failure and timeout behaviour

The main current validation targets are:

external Smagorinsky
    ↔ standard OpenFOAM Smagorinsky

external kEqn
    ↔ standard OpenFOAM kEqn

---

20. Development

Run the focused FoamPilot tests with:

cd /path/to/SmartSim-CSC

PYTHONPATH="$PWD/components/foampilot" \
python3 -m pytest \
    tests/openfoam \
    -q

Build the package with:

cd components/foampilot

rm -rf build dist *.egg-info

python3 -m build
python3 -m twine check dist/*

Before committing:

git diff --check
git status --short

git add components/foampilot
git diff --cached --stat
git diff --cached --check
git diff --cached

---

21. Design principle

FoamPilot follows one central design rule:

Keep OpenFOAM responsible for mesh-aware finite-volume computation, and make external models responsible for clearly typed closure calculations.

This keeps the integration narrow enough to validate, while still allowing NumPy, JAX, ONNX, and machine-learning models to participate directly in an OpenFOAM simulation loop.

---

Licence

FoamPilot CSC is distributed under the licence included with the SmartSim-CSC source distribution. The bundled OpenFOAM, SmartSim, SmartRedis, RedisAI, and other upstream components retain their respective licence and attribution requirements.