LbAPCommon

Common utilities for LHCb DPA WP2 related software, including Analysis Productions workflow parsing, validation, and conversion tools.

Features

Workflow Parsing: Parse and render Analysis Productions YAML configuration files
DIRAC Conversion: Convert workflow definitions to DIRAC production requests
CWL Conversion: Convert production requests to Common Workflow Language (CWL) format
Validation & Linting: Validate workflow configurations and check for common issues
Data Models: Pydantic v2 models for workflow and production data structures

Installation

pip install LbAPCommon

For development:

# Using pixi (recommended)
pixi install
pixi run test

# Using conda/mamba
mamba env create --file environment.yaml
conda activate lbaplocal-dev
pip install -e '.[testing]'

Usage

DIRAC Production Request Generation

Convert Analysis Productions info.yaml files to DIRAC production requests. This is the primary CLI used by the Analysis Productions system:

python -m LbAPCommon <production_name> --input info.yaml --output output.json

# With specific AP package version
python -m LbAPCommon my_production --input info.yaml --output result.json --ap-pkg-version v1r0

# Process only a specific job from the workflow
python -m LbAPCommon my_production --input info.yaml --output result.json --only-include job_name

# Dump individual YAML request files for each production
python -m LbAPCommon my_production --input info.yaml --output result.json --dump-requests

# With DIRAC server credentials
python -m LbAPCommon my_production --input info.yaml --output result.json --server-credentials user password

Arguments:

production_name - Name of the production (alphanumeric + underscore, 2-200 chars)
--input - Path to the info.yaml workflow definition file
--output - Path for the output JSON file containing production requests
--ap-pkg-version - AnalysisProductions package version (default: v999999999999)
--only-include - Only process the workflow chain containing this job name
--dump-requests - Also write individual .yaml files for each production request
--server-credentials - DIRAC server credentials (username and password)

The output JSON contains:

rendered_yaml - The rendered workflow YAML after Jinja2 processing
productions - Dictionary of production requests, each containing:
- request - The DIRAC production request in LbAPI format
- input-dataset - Input dataset specification from Bookkeeping query
- dynamic_files - Auto-generated configuration files
- raw-yaml - Original YAML for the jobs in this production

CWL Conversion CLI

Convert DIRAC production request YAML files to CWL workflows:

# List productions in a file
lhcb-production-yaml-to-cwl list_productions production.yaml

# Generate CWL workflow files
lhcb-production-yaml-to-cwl generate production.yaml --output-dir ./cwl-output

# Convert a specific production
lhcb-production-yaml-to-cwl generate production.yaml --production "MyProduction"

Python API

from LbAPCommon import parse_yaml, render_yaml, validate_yaml
from LbAPCommon.dirac_conversion import group_in_to_requests, step_to_production_request
from LbAPCommon.prod_request_to_cwl import fromProductionRequestYAMLToCWL

# Parse and validate workflow YAML
rendered = render_yaml(yaml_content)
jobs_data = parse_yaml(rendered, production_name, wg)
validate_yaml(jobs_data, wg, production_name)

# Group jobs into production requests
for job_names in group_in_to_requests(jobs_data):
    request = step_to_production_request(
        production_name, jobs_data, job_names, input_spec, ap_pkg_version
    )

# Convert DIRAC output to CWL
workflow, inputs, metadata = fromProductionRequestYAMLToCWL(yaml_path)

Development

Running Tests

pixi run test              # Run all tests
pixi run test-dirac        # Run DIRAC conversion tests
pixi run test-cov          # Run with coverage report

Updating Test Fixtures

When CWL output changes, update reference fixtures:

pixi run update-fixtures   # Regenerate reference .cwl files

Code Quality

pixi run pre-commit        # Run all pre-commit hooks (linting, formatting, etc.)

Documentation

Documentation goes in the LbAPDoc project.

LbAPI - REST API server for Analysis Productions
LbAPLocal - Local testing tools
LbAPWeb - Web interface for Analysis Productions

License

GPL-3.0 - See COPYING for details.

Metadata

Release files for LbAPCommon 0.19.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for LbAPCommon 0.19.1
File	Size	Uploaded
lbapcommon-0.19.1.tar.gz	533.9 kB	Jul 13, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for LbAPCommon 0.19.1
File	Interpreter	ABI	Platform	Reset
lbapcommon-0.19.1-py3-none-any.whl 182.0 kB Jul 13, 2026	Python 3	none	any	Details

Total release size: 715.9 kB

Release files / lbapcommon-0.19.1.tar.gz

Download URL	lbapcommon-0.19.1.tar.gz
Size	533.9 kB
Tags	Source
SHA-256 checksum How to use checksums	`e3797a7548317ca18d65c22e4fa59cfaf61e962240ea027e8b91d6dfd74deae8`
BLAKE2b-256 checksum How to use checksums	`5a0bc0ef02077a7838f9e7137fe70475f2cce97dadef3ab772b90e69d3c7e343`
Upload date	Jul 13, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/6.2.0 CPython/3.9.25`

Release files / lbapcommon-0.19.1-py3-none-any.whl

Download URL	lbapcommon-0.19.1-py3-none-any.whl
Size	182.0 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`f11506193e49265c64b4d6cdc853ea8b397eec9648fce68fb28e192b5d95f154`
BLAKE2b-256 checksum How to use checksums	`8d5bc47f6b1341df22c119fc7b8688d4a3654237a8f63dbd208c389ddc015361`
Upload date	Jul 13, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/6.2.0 CPython/3.9.25`