cp2kpal 0.1.0

Reusable CP2K workflow infrastructure for local, Docker, and Slurm execution.

cp2kpal — Reusable CP2K Workflow Infrastructure

cp2kpal is a Python library and CLI for managing CP2K simulations across local, Docker, and Slurm-based HPC environments. It provides reusable infrastructure for configuration, execution, provenance tracking, and research project organization, with the science-specific workflows kept in downstream projects.

Features

• Research management — Organize work into projects, experiments, and runs via filesystem-backed CRUD (cp2kpal research ...)
• Configuration management — Pydantic-validated YAML configs with strict schema enforcement
• Execution backends — Local subprocess, Docker container, and Slurm sbatch generation/submission
• Job tracking — sacct-based status queries and job metadata syncing
• Work directory management — Stage creation, artifact protection, slug/hash utilities
• Provenance tracking — Git-based versioning, file hashing, stage metadata serialization
• CLI output utilities — Structured error handling, table/JSON/CSV output formatting

Installation

pip install cp2kpal

Or with uv (recommended):

uv add cp2kpal

Requires Python ≥3.11, <3.12.

Quick Start — Python API

Load a config and prepare a work directory

from pathlib import Path
from cp2kpal.config import ProjectSettings, load_config, create_stage_dirs

config = load_config("config.yaml", ProjectSettings)
dirs = create_stage_dirs(config.work_dir / "run_001" / "scf")
print(f"Inputs: {dirs['inputs']}")
print(f"Logs:  {dirs['logs']}")

Generate and submit a Slurm job

from cp2kpal.config import SlurmSettings
from cp2kpal.exec import render_sbatch, write_sbatch, submit_sbatch

settings = SlurmSettings(partition="deimos_l", ntasks=120, walltime="16:00:00")
script = render_sbatch(
    settings,
    job_name="cp2k-run",
    command="mpiexec -n 120 cp2k.psmp -i input.inp -o output.out",
)
script_path = write_sbatch(Path("logs/submit.sbatch"), script)
result = submit_sbatch(script_path)

Run a local CP2K process

from cp2kpal.exec import local_run

returncode, stdout, stderr = local_run(
    command="cp2k.psmp -i input.inp -o output.out",
    workdir=Path("work/run_001"),
    timeout=3600,
)

Track job status via sacct

from cp2kpal.exec import sacct_status, checked_at

jobs = sacct_status(job_ids=["123456", "123457"])
print(f"Checked at: {checked_at()}")
for job in jobs:
    print(f"  {job['job_id']}: {job['state']} ({job['elapsed']})")

Quick Start — CLI (Research Management)

cp2kpal provides a CLI for organizing computational work into a hierarchical structure:

research/
└── <project>/
    ├── .cp2kpal.yaml          # Project metadata
    ├── analysis/              # Analysis code and outputs
    │   └── outputs/           # Generated figures (git-ignored)
    └── exps/
        └── <experiment>/
            ├── .cp2kpal.yaml  # Experiment metadata (incl. tags)
            ├── config.yaml    # Experiment-specific config
            └── runs/           # Run results (git-ignored)
                ├── job_xxx/   # HPC download data
                └── run_xxx/   # Pipeline outputs

Projects

# List all projects
cp2kpal research list

# Create a new project
cp2kpal research create my_project --description "My research project"

# Show project details (including experiments and run counts)
cp2kpal research show my_project

# Delete a project (--force for non-empty)
cp2kpal research delete my_project --force

Experiments

# List experiments in a project
cp2kpal research exp list my_project

# Create an experiment with tags
cp2kpal research exp create my_project exp_001 \
    --description "Initial calculation" \
    --tag "angle=21.7868" --tag "method=CI-NEB"

# Show experiment details
cp2kpal research exp show my_project exp_001

# Delete an experiment
cp2kpal research exp delete my_project exp_001 --force

Runs

# List runs in an experiment
cp2kpal research run list my_project exp_001

# Create an empty run directory
cp2kpal research run create my_project exp_001 my_run \
    --description "Test run" --source "manual"

# Import HPC download as a new run (copies data)
cp2kpal research run import my_project exp_001 /path/to/hpc/output \
    --run-id job_5361556

# Show run details
cp2kpal research run show my_project exp_001 job_5361556

# Delete a run
cp2kpal research run delete my_project exp_001 my_run --force

By default, cp2kpal auto-detects a research/ directory by walking up from the current working directory (looking for a parent with .git or pyproject.toml). Override with $CP2KPAL_RESEARCH_DIR or --research-dir.

Modules

Module	Description
cp2kpal.config	Pydantic-validated YAML config models (ProjectSettings, SlurmSettings, DockerSettings, Cp2kSettings)
cp2kpal.exec	Execution backends: local_run, render_sbatch, write_sbatch, submit_sbatch, sacct_status
cp2kpal.storage	Work directory layout: create_stage_dirs, run_dir, stage_dir, artifact protection
cp2kpal.provenance	Hashing (file_hash, config_hash), git version info (git_sha, git_status), stage metadata
cp2kpal.research	Research project/experiment/run CRUD via filesystem-backed store
cp2kpal.cli	CLI output helpers: CliError, fail, print_data (table/JSON/CSV)

Development

git clone <repo>
cd packages/cp2kpal
uv sync --extra dev
uv run pytest

Format and lint:

uv run ruff format src/
uv run ruff check src/

Design Philosophy

• Infrastructure, not science — cp2kpal provides reusable primitives; domain-specific science, CP2K templates, and result parsers belong in downstream projects.
• Filesystem-backed, no database — The research management module stores metadata as YAML files in a standard directory layout, keeping things git-friendly and zero-dependency.
• Single source of truth — Downstream projects (e.g. tbg-proton) import cp2kpal.research directly rather than shelling out, ensuring consistent behavior.

License

MIT