Parallel Computing Validation System (PCVS) is a Validation Orchestrator designed by and for software at scale. Its primary target is HPC applications & runtimes but can flawlessly address smaller use cases. PCVS can help users to create their test scenarios and reuse them among multiples implementations, a high value when it comes to validating programming standards (like APIs & ABIs). No matter the number of programs, benchmarks, languages, or tech non-regression bases use, PCVS gathers in a single execution, and, with a focus on interacting with HPC batch managers efficiently, run jobs concurrently to reduce the time-to-result overall.

Quick installation guide

PCVS is a Python program distributed on PyPI. It can be easily installed using pip.

$ pip3 install pcvs

Alternatively, you can install and test it from source.

# Considering python3.10+
$ pip3 install .
# For dev/testing purposes, use the following to install development dependencies
$ pip3 install '.[dev]'
# Basic tests
$ tox -e pcvs-coverage
# OR
$ coverage run

Once installed, the program should be available as pcvs.

Basic usage

Profiles and test description

PCVS relies on two YAML-based sets of configurations to understand the environment and the suite.

• The profiles describe the testing environment (compilers, machine, criterions...). They are split in multiple files to help compose profile from basic blocks. They can be manipulated through the pcvs config subcommand. For more details, you can refer to the documentation.

• The test descriptions describe the test suite (how to build, run and validate). They are independent from the profiles to help share suites adapted to PCVS across different platforms. They can be static (a plain YAML file) or dynamic (generated on the fly by a script). A complete example of a test description can be found in the documentation.

The validity of the YAML files can be asserted using the pcvs check subcommand.

Run a test suite

Once the configurations setup, they can be reused indefinitely to run tests.

$ pcvs run -p <profile> <path/to/suite>

Using the above command will launch PCVS on a given test suite, building and validating it. A quick summary is available at the end of the run.

Visualize the results

Once the PCVS run completed, there are two ways to visualize the results:

• Using the webview with pcvs report subcommand and opening localhost:5000 in a browser

• Using the TUI with pcvs --tui report

The output of the different tests can be retrieved from either as well as the complete command ran.

Complete documentation

The user guide is hosted on GitHub pages: https://cea-hpc.github.io/pcvs. Every CLI command has a builtin help that can be displayed using the -h option. Alternatively, the documentation can be generated locally from the source code.

Authors

This work is currently supported by the French Alternative Energies and Atomic Energy Commission (CEA). For any question and/or remarks, please contact:

• Hugo TABOADA hugo.taboada@cea.fr
• Nicolas MARIE nicolas.marie@uvsq.fr

Licensing

PCVS is distributed under the terms of the CeCILL-C Free Software Licence Agreement.

All new contributions must be made under the CeCILL-C terms.

For more details, see COPYING or CeCILL-C website.

SPDX-Licence-Identifier: CECILL-C