digout 0.1.3

digout is a Python library purpose-built to execute the multi-stage workflow of converting raw LHCb DIGI files into analysis-ready parquet dataframes of particles and hits.

To manage this process in a scalable and reproducible manner, it implements a workflow framework organized around configurable steps (e.g., digi2root, root2df). The framework operates on a two-phase execution model: a stream phase runs once to prepare the dataset from a bookkeeping path, and a chunk phase processes each input file in parallel. This parallel execution is managed by swappable schedulers (such as local for local processing or htcondor for cluster submission), with the entire workflow being defined through YAML configuration files to ensure complete reproducibility.

Resources

Core Features

• Automated Metadata Discovery: Automatically queries the LHCb bookkeeping system to retrieve necessary metadata (dddb_tag, conddb_tag, etc.), eliminating manual lookup.
• Scalable Parallel Processing: Built-in support for processing large datasets in parallel on a local machine or on a distributed cluster like HTCondor.
• Configuration-Driven and Reproducible: Define your entire workflow in YAML files. digout saves the final, resolved configuration for every run, ensuring any result can be reproduced.
• Idempotent Execution: Automatically detects and skips steps that have already been completed.
• Extensible Architecture: Easily define new steps or schedulers.

Main Workflows

• DIGI to DataFrame Conversion: Produce analysis-ready parquet dataframes from LHCb DIGI files. The available output dataframes are detailed on the DataFrames Page.

• DIGI to MDF Conversion: Convert LHCb DIGI files into the .mdf format required as input for the Allen framework.