Sparkkflow

ARPES workflow engine with SPR-KKR, OSCARpes ingestion and ML polarization calibration.

Install

pip install -e .              # core
pip install -e ".[ml]"        # + torch / scikit-learn
pip install -e ".[oscarpes]"  # + OSCARpes ingestion
pip install -e ".[dev]"       # + pytest / ruff / mypy

CLI

sparkkflow doctor                                   # check tooling
sparkkflow scan --energy-range 20 50 --dicho        # run an ARPES scan
sparkkflow generate --energy-range 20 100 --fine-grid
sparkkflow train --model-path pol.pth
sparkkflow predict --model-path pol.pth --energy 30
sparkkflow monitor --job-ids 12,34 --continuous

Library

from sparkkflow.config import manager
from sparkkflow.scans import scans
from sparkkflow.jobs import submitter
from sparkkflow.ml import pipeline, train, predict
from sparkkflow.monitor import watch, status

cfg = manager("config.yaml").config
runner = scans(scheduler_type=cfg.scheduler.default_type, ml_mode=False)
runner.run(energy_range=(20, 50), dicho=True)

Layout

sparkkflow/
  __init__.py       # public API surface
  cli.py            # `sparkkflow` console entry point
  config.py         # YAML loader, env overrides, validation
  log.py            # structured logging
  jobs.py           # SLURM / SGE submitter (`submitter`, `slurm`, `sge`)
  scans.py          # ARPES parameter scans (`scans`)
  ml.py             # PyTorch pipeline + train / predict
  monitor.py        # status polling and resubmission
  plot.py           # dichroism plotting
  trcdad*.py        # TR+CDAD utilities
  calc_*.py         # SPR-KKR calculators
  atom_*.py         # atomic-position scans
config.yaml         # configuration template
examples/           # runnable demos
tests/              # pytest suite

Naming conventions

• All modules and public classes are lowercase, snake_case, ASCII.
• No +, -, spaces, or version words like enhanced in filenames.
• One short responsibility per module name (config, jobs, ml, scans).

Development

pytest -q
ruff check sparkkflow tests
mypy sparkkflow