Skip to main content

fasthep-curator

CI PyPI Python Versions Documentation Status Discussions

FAST-HEP

fasthep-curator provides dataset inspection, metadata extraction, validation, and schema generation utilities for FAST-HEP workflows.

The Python import namespace is:

import fasthep_curator

Scope

fasthep-curator is responsible for:

  • dataset inspection
  • branch discovery
  • schema generation
  • metadata extraction
  • validation helpers
  • workflow diagnostics
  • runtime error reporting helpers
  • caching of inspection artifacts

It is the dataset and metadata management layer of the FAST-HEP ecosystem.

Relationship to fasthep-flow

fasthep-flow provides:

  • workflow compilation
  • orchestration
  • execution planning
  • backend interfaces

fasthep-curator provides:

  • dataset metadata
  • source schemas
  • validation utilities
  • dataset inspection tooling
  • diagnostics and reporting helpers

In practice, most HEP users will use both packages together.

  • fasthep-flow

    • workflow language and execution engine
  • fasthep-carpenter

    • HEP analysis transforms
    • histogramming
    • event processing
  • fasthep-render

    • plotting
    • tables
    • reports
  • fasthep-cli

    • the unified fasthep command-line interface

Alternatively, install the meta package:

pip install fasthep

Installation

Install directly:

pip install fasthep-curator

Development environment:

pixi install
pixi run ci

Minimal example

Example dataset inspection:

datasets:
  DYJets:
    files:
      - /data/example.root
    source:
      type: root_tree
      tree: Events

Example metadata artifact:

source_schema:
  events:
    pt: float
    eta: float
    phi: float

ROOT Schema Inspection

Curator provides the reusable Python schema-inspection API. Command-line access is exposed through the central FAST-HEP CLI.

Inspect a ROOT TTree interactively:

fasthep inspect schema input.root --tree Events

The default output is a readable table with field name, logical dtype, shape, and ROOT storage type.

Extract a pasteable YAML list for keep: or drop: workflow parameters:

fasthep inspect schema input.root \
  --tree Events \
  --include 'GenJet*' \
  --include 'GenMET*' \
  --include 'Generator_*' \
  --format yaml-list

Generate a starter hep.align_schema document:

fasthep inspect schema input.root \
  --tree Events \
  --include 'GenJetAK8_*' \
  --format alignment

Field filters use shell-style glob matching. Repeated --include options are unioned in source field order; repeated --exclude options remove matches after includes:

fasthep inspect schema input.root \
  --tree Events \
  --include 'Gen*' \
  --exclude 'GenPart_*'

The same inspection implementation is available from Python:

from fasthep_curator.api import inspect_root_tree_schema

schema = inspect_root_tree_schema("input.root", tree="Events")

Dataset Entry Counts (compile hook)

The dataset_metadata.root_tree compile hook records total_entries per dataset in dataset_metadata.json. By default the count comes from the dataset's nevents key, and no file is opened. A dataset without a valid nevents (an integer, or a digit-only string, not negative) is read from its files instead, and total_entries_fallback says why.

Parameters (set under params of the registry entry):

Parameter Default Effect
total_entries_source nevents Dataset key that holds the count. null reads the files of every dataset.
add_per_file false Also record entries per file under files (opens every file). Warns when the sum differs from the declared count.
max_files_warn 100 One warning, before any file is opened, when more files than this are about to be read.

total_entries_origin records the key that was used, or read. Unknown parameter names are an error.

Limitation: a count taken from nevents is not checked against the files list. A list that was trimmed after nevents was written is inconsistent input that the hook cannot detect; keep both from the same source (for example per-file counts from DAS), or use add_per_file once to compare them.

Design principles

fasthep-curator focuses on:

  • reproducible metadata
  • reusable inspection artifacts
  • declarative validation
  • cached schema discovery
  • workflow diagnostics
  • experiment-agnostic interfaces where possible

The package intentionally separates dataset/metadata management from workflow orchestration and analysis execution.

Documentation

Main FAST-HEP documentation:

API documentation for this package:

Repository

Main FAST-HEP repository and project links:

Contributing

Contribution guidelines, development setup, and project-wide documentation are maintained centrally in the main FAST-HEP repository.

Legacy branch

Earlier prototype implementations are preserved in legacy repositories and branches.

The current repository contains the split-package FAST-HEP architecture.

Status

FAST-HEP is currently in active pre-alpha development.

Interfaces may evolve rapidly while the package split and stabilization work continues.

Metadata

Release files for fasthep-curator 2026.10.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for fasthep-curator 2026.10.1
File Size Uploaded Provenance Actions
fasthep_curator-2026.10.1.tar.gz 20.5 kB With provenance Details for fasthep_curator-2026.10.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for fasthep-curator 2026.10.1
File Interpreter ABI Platform Actions
fasthep_curator-2026.10.1-py3-none-any.whl Python 3 none any Details for fasthep_curator-2026.10.1-py3-none-any.whl

Total release size: 47.1 kB

Release files / fasthep_curator-2026.10.1.tar.gz

Download URL fasthep_curator-2026.10.1.tar.gz
Size 20.5 kB
Tags Source
SHA-256 checksum
How to use checksums
b311560fba55762fb6676d2b7062b74a90efb2b0b939b9e266bf5961b15ff010
BLAKE2b-256 checksum
How to use checksums
a3b51b3f0a850b514edbc9df93ccd3f278c8f5ce0bd8db936fd6090652ffbaf1
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 26, 2026.

Transparency log

Release files / fasthep_curator-2026.10.1-py3-none-any.whl

Download URL fasthep_curator-2026.10.1-py3-none-any.whl
Size 26.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
2db21872af1598981cc9e4c6791c327b84bd960958973f11bbca46e96d8e6698
BLAKE2b-256 checksum
How to use checksums
18f2845cbaf66c29d36c4887b89ea5dd6f46cada5aed6b14c14ee69b335e60e6
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 26, 2026.

Transparency log

Security for fasthep-curator 2026.10.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for fasthep_curator-2026.10.1.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 26, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page