Skip to main content

ReFrame is a powerful framework for writing system regression tests and benchmarks, specifically targeted to HPC systems

Project description

ReFrame Logo
Build Status Documentation Status codecov.io
GitHub release (latest by date including pre-releases) GitHub commits since latest release GitHub contributors
PyPI version PyPI - Python Version Downloads Downloads
Slack
License DOI
Twitter Follow

ReFrame in a Nutshell

ReFrame is a powerful framework for writing system regression tests and benchmarks, specifically targeted to HPC systems. The goal of the framework is to abstract away the complexity of the interactions with the system, separating the logic of a test from the low-level details, which pertain to the system configuration and setup. This allows users to write portable tests in a declarative way that describes only the test's functionality.

Tests in ReFrame are simple Python classes that specify the basic variables and parameters of the test. ReFrame offers an intuitive and very powerful syntax that allows users to create test libraries, test factories, as well as complete test workflows using other tests as fixtures. ReFrame will load the tests and send them down a well-defined pipeline that will execute them in parallel. The stages of this pipeline take care of all the system interaction details, such as programming environment switching, compilation, job submission, job status query, sanity checking and performance assessment.

Please visit the project's documentation page for all the details!

Installation

ReFrame is fairly easy to install. All you need is Python 3.6 or above and to run its bootstrap script:

git clone https://github.com/reframe-hpc/reframe.git
cd reframe
./bootstrap.sh
./bin/reframe -V

If you want a specific release, please refer to the documentation page.

Running the unit tests

You can optionally run the framework's unit tests with the following command:

./test_reframe.py -v

NOTE: Unit tests require a POSIX-compliant C compiler (available through the cc command), as well as the make utility.

Building the documentation locally

You may build the documentation of the master manually as follows:

./bootstrap.sh +docs

For viewing it, you may do the following:

cd docs/html
python3 -m http.server

The documentation is now up on localhost:8000, where you can navigate with your browser.

Test library

The framework comes with a library of tests that users can either run them from the command line directly or extend them and fine tune them for their systems. See here for more details.

Test examples

You can find examples of real tests under the ReFrame HPC community Github page. The most complete suite of tests currently publicly available is that of CSCS, which you can also find here. You can use those tests as a starting point for implementing your own tests.

Contact

You can get in contact with the ReFrame community in the following ways:

Slack

Please join the community's Slack channel for keeping up with the latest news about ReFrame, posting questions and, generally getting in touch with other users and the developers.

Mailing list

You may also subscribe to the mailing list. Only subscribers can send messages to the list. For unsubscribing, you may send an empty message here.

Contributing back

ReFrame is an open-source project and we welcome and encourage contributions! Check out our Contribution Guide here.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ReFrame-HPC-3.12.0.tar.gz (207.9 kB view hashes)

Uploaded source

Built Distribution

ReFrame_HPC-3.12.0-py3-none-any.whl (243.2 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page