fasthep-carpenter

fasthep-carpenter provides common analysis building blocks for FAST-HEP workflows.

It contains reusable High Energy Physics transforms, sources, sinks, and runtime helpers built on top of fasthep-flow.

The Python import namespace is:

import fasthep_carpenter

Scope

fasthep-carpenter is responsible for:

• ROOT and awkward-array based sources
• event stream manipulation
• HEP analysis transforms
• histogram filling
• cutflows
• object selection helpers
• common CMS/LHC analysis utilities
• workflow runtime extensions

It is the main “analysis implementation” layer of the FAST-HEP ecosystem.

Relationship to fasthep-flow

fasthep-flow provides:

• workflow compilation
• execution planning
• orchestration
• registries
• backend interfaces

fasthep-carpenter provides:

• concrete analysis operations
• HEP-specific runtime behaviour
• physics object manipulation
• common workflow primitives

In practice, most HEP users will use both packages together.

Recommended companion packages

• fasthep-flow

  • workflow language and execution engine

• fasthep-curator

  • dataset inspection
  • schema generation
  • metadata snapshots

• fasthep-render

  • plotting
  • tables
  • reports

• fasthep-cli

  • the fasthep command-line interface

Alternatively, install the meta package:

pip install fasthep

Installation

Install directly:

pip install fasthep-carpenter

Development environment:

pixi install
pixi run ci

Minimal example

Example transform registration:

registry:
  transforms:
    define:
      spec: fasthep_carpenter.spec.define_transform:DEFINE_TRANSFORM_SPEC
      impl: fasthep_carpenter.impl.define_transform:run_define_transform

Example workflow snippet:

steps:
  - id: TightMuon
    op: hep.select_objects
    params:
      collection: Muon
      output: selected_tight_Muon
      selection:
        - pt >= 20
        - abs(eta) <= 2.4
      keep:
        - pt
        - eta
        - phi
        - mass

Schema Alignment

hep.align_schema aligns an event stream to an explicit logical schema before a writer or downstream compatibility boundary. It handles projection, renaming, and intentional dtype casts on Awkward fields. It does not control ROOT serialization details such as TTree/RNTuple choice, compression, basket layout, or physical branch splitting.

Inline schema:

- id: AlignSmallSchema
  op: hep.align_schema
  params:
    schema:
      version: 1
      fields:
        ndiMuon_Z:
          dtype: int32
        legacy_name:
          source: fasthep_name
          dtype: float32

External YAML or JSON schema:

- id: AlignLegacySchema
  op: hep.align_schema
  params:
    schema: validation/schemas/hinv_legacy_dtype_compat.yaml
    missing: error
    extra: drop

External schemas are loaded by fasthep-flow while the workflow is normalized and compiled. The runtime operation receives the resolved schema mapping; it does not reopen the YAML or JSON file during event processing.

For gradual migration, keep unresolved and unmentioned fields while the target schema is incomplete:

params:
  schema: validation/schemas/partial.yaml
  missing: ignore
  extra: keep

For a strict external contract, require every source field and drop extras:

params:
  schema: validation/schemas/contract.yaml
  missing: error
  extra: drop

hep.select_objects evaluates selection expressions relative to the input collection, keeps exactly the configured fields, emits n<output> as the selected-object count, and sorts selected objects by descending pt by default. Use sort to override the ordering or sort: false to preserve input order. Overlap removal is intentionally separate and belongs in hep.clean.

Use hep.build_pairs when an analysis needs explicit pair-candidate products rather than a first-two-object scalar mass. The operation accepts one or more input collections, concatenates them in declared order, forms all unordered pairs, evaluates pair expressions in a lepton_1_<field> / lepton_2_<field> context, builds candidate four-vectors, evaluates candidate expressions in a pt/eta/phi/mass context, and can stably sort the candidate and aligned constituent collections without choosing or truncating to one candidate. It emits explicit flat output fields plus n<output>_Z; use hep.selection.flag on that count when a reusable event flag is needed. hep.di_object_mass remains the simpler operation for a scalar mass from the first two objects in one collection.

Use hep.build_lepton_met_candidate for single-lepton plus MET candidates. It consumes one lepton collection and one scalar MET product, evaluates lepton selection expressions relative to the lepton collection, broadcasts MET against all selected leptons, and writes aligned <output>_W_* and <output>_lepton_* collections. The candidate context exposes pt, eta, phi, mass, and MT, where MT is the transverse mass. Counts are explicit products: n<output>_lepton records the number of leptons after lepton selection, while n<output>_W records the number of candidates after candidate selection. The operation preserves all surviving candidates and does not filter events; use hep.selection.flag on the count products for reusable event booleans.

Use hep.build_recoil for transverse recoil candidates built from scalar MET and zero or more already aligned visible-object collections. It consumes <met>_pt, <met>_phi, and each visible collection's pt/phi, computes MET + visible transverse components, and writes configured recoil fields plus n<output>. Selection expressions are evaluated on the generated recoil fields such as pt and phi; they do not add source-branch dependencies. The default reduction records the count before reduction and keeps the highest-pt recoil candidate without filtering events.

Use hep.choose when several event categories choose between coherent sets of event-level values. This replaces hard-to-scan nested conditionals such as:

- name: Recoil_pt
  expr: where(SingleMuon_CR_selection, singleMuon_recoil_pt,
        where(DiMuon_CR_selection, diMuon_recoil_pt,
        where(SR_selection, SR_recoil_pt, -999.0)))
- name: Recoil_phi
  expr: where(SingleMuon_CR_selection, singleMuon_recoil_phi,
        where(DiMuon_CR_selection, diMuon_recoil_phi,
        where(SR_selection, SR_recoil_phi, -999.0)))

Nested conditionals hide priority in expression structure, repeat the same choice separately for related outputs, silently choose the first match when selections overlap, and make accidental scalar/jagged mixing easy. The same logic can be written explicitly:

- id: ChooseRecoil
  op: hep.choose
  params:
    cases:
      - name: singleMuon
        when: SingleMuon_CR_selection
        values:
          Recoil_pt: leading(singleMuon_recoil_pt, -999.0)
          Recoil_phi: leading(singleMuon_recoil_phi, -999.0)
      - name: diMuon
        when: DiMuon_CR_selection
        values:
          Recoil_pt: leading(diMuon_recoil_pt, -999.0)
          Recoil_phi: leading(diMuon_recoil_phi, -999.0)
      - name: signalRegion
        when: SR_selection
        values:
          Recoil_pt: leading(SR_recoil_pt, -999.0)
          Recoil_phi: leading(SR_recoil_phi, -999.0)
    default:
      Recoil_pt: -999.0
      Recoil_phi: -999.0
    on_multiple: error
    on_no_match: default

Each when predicate must produce one boolean per event. Each value expression must produce one scalar value per event, or be a scalar expression that can be broadcast to all events. All cases, and the default mapping when on_no_match: default is used, must define exactly the same output names. Use on_multiple: error to require mutually exclusive cases, or on_multiple: first to make authored-order priority explicit. Use on_no_match: error to require a category for every event, or on_no_match: default to fill unmatched events from the default mapping. Cases are evaluated in authored order only when first semantics are requested.

Use hep.selection.cutflow for event filtering and cutflow accounting. An ordinary selection expression must be an event-level boolean expression that produces exactly one decision per event, for example MET_pt > 120. Object or jagged expressions must be reduced explicitly, for example {reduce: {op: any, over: "Muon_Pt > 25"}} or {reduce: {op: all, over: "Jet_passId"}}. A singleton object collection is still jagged and is not automatically treated as an event scalar; reduce it explicitly or convert it to an event-level field upstream.

Use hep.selection.flag for event-level predicates that should be materialized as boolean fields without filtering events or producing cutflow counts. Its selection list is combined with logical AND and written to output; if output is omitted, workflow normalization fills it from the exact stage id. Expressions operate on event products directly, so object-count predicates should reference the conventional count field, for example ncleaned_veto_Electron == 0.

Design principles

fasthep-carpenter focuses on:

• reusable analysis primitives
• declarative workflows
• registry-driven extension
• experiment-agnostic interfaces where possible
• compatibility with awkward-array based analysis ecosystems

The package intentionally separates workflow orchestration (fasthep-flow) from domain-specific analysis behaviour.

Documentation

Main FAST-HEP documentation:

• https://fast-hep.github.io

API documentation for this package:

• https://fasthep-carpenter.readthedocs.io/en/latest/

Repository

Main FAST-HEP repository and project links:

• https://github.com/FAST-HEP/fasthep

Contributing

Contribution guidelines, development setup, and project-wide documentation are maintained centrally in the main FAST-HEP repository.

Legacy branch

The pre-split prototype implementation is preserved in the legacy branch.

The new main branch contains the split-package architecture.

Status

FAST-HEP is currently in active pre-alpha development.

Interfaces may still evolve rapidly while the package split and stabilization work continues.