hepyy-workflows

An explicit HEP workflow package. SPECS.md defines the full project; this checkout currently implements milestones 1 through 6. The package name is hepyy_workflows and the CLI is hwf. Python 3.11+ is required.

The package provides Arrow dataset/source contracts, strict public reader and recipe schemas, a starter project, and bounded inspect/draft/validate/preview workflows for Parquet, ROOT TTrees, explicit HDF5 layouts, and HepMC3 ASCII. Recipes now run generic views, selections, histograms and cutflows and export a standalone file-analysis runner, self-contained reports, and local web/terminal authoring tools.

henv -n YOUR_ENV -x python -m pip install -e '.[dev]'
henv -n YOUR_ENV -x python -m pytest

Create a starter project from any directory (choose a destination that does not yet exist):

henv -n YOUR_ENV -x hwf project init /path/to/my-analysis
cd /path/to/my-analysis
henv -n YOUR_ENV -x hwf schema validate reader reader.yaml --json
henv -n YOUR_ENV -x hwf schema validate recipe recipe.yaml --json
henv -n YOUR_ENV -x hwf reader inspect data/particles.parquet --json
henv -n YOUR_ENV -x hwf reader validate reader.yaml --input data/particles.parquet --json
henv -n YOUR_ENV -x hwf reader preview reader.yaml --input data/particles.parquet --events 2 --json
henv -n YOUR_ENV -x hwf recipe run recipe.yaml --input data/particles.parquet --output results --json
henv -n YOUR_ENV -x hwf recipe export recipe.yaml --input data/particles.parquet --output standalone --json

The last command reads the starter project's sample file. See working-in-a-project.md for what the starter files mean and examples/readers for each format. For choosing event boundaries, collection paths, units and labels in an unfamiliar file, see FILE_FORMAT_README.md. See analysis.md for recipe operations, histogram products, and standalone execution; the Lund and dilepton examples are complete workflows.

Build an offline report from completed results, or inspect them remotely in a terminal:

henv -n YOUR_ENV -x hwf report results --output results.html --json
henv -n YOUR_ENV -x hwf tui results
henv -n YOUR_ENV -x hwf web recipe.yaml --port 8765

The report is one HTML file with embedded data and scripts; it works from file:// without a server or network connection. The web editor listens on localhost and validates edits through the same service as the CLI.

Agents can use the same operations through line-delimited JSON:

printf '%s\n' '{"id":1,"method":"inspect_input","params":{"input":"data/particles.parquet"}}' \
  | henv -n YOUR_ENV -x hwf agent serve
henv -n YOUR_ENV -x hwf reader scaffold my_reader --output src/my_reader

See agent-workflow.md for the typed method contract and the recommended inspect → draft → validate → preview → run → export flow.

Recipes may declare a visible Python preprocessing module. It receives the ordered input paths and a Parquet output path, and its source and dependencies are bundled by standalone export. See processing.md.

For an LLM-assisted jet-observable workflow and a complete angularities example, see HOW-TO-EXAMPLE-WORKFLOW.md.

To compare two completed results at matched quantiles of jet pT (Brewer, Milhano and Thaler's procedure for Pb-Pb versus pp), use hwf quantile match; see quantile-matching.md and the quantile example.

hwf schema validate accepts reader-draft, reader, or recipe; failed validation exits nonzero and reports stable diagnostic codes and YAML paths. Paths passed to the CLI are resolved from your current directory; use absolute paths when invoking it from elsewhere. A missing file reports FILE_NOT_FOUND. An omitted momentum-field unit resolves to GeV/c and appears in assumed_units in JSON output. Declare a different source unit explicitly when needed. Schema files are available under schemas/ and ship inside the wheel.

The package requires NumPy, PyArrow, JSON Schema and PyYAML. Pure histogramming does not import hepyy, FastJet, PYTHIA, HepMC, uproot, h5py or a web server. Optional extras in pyproject.toml reserve these dependency boundaries. For all milestone 2 readers, install .[root,hdf5,hepmc] in the analysis environment. See architecture.md and provenance.md.