rEproducible sofTware peRformance analysIs in perfeCt Simplicity
Project description
mETRICS - rEproducible sofTware peRformance analysIs in perfeCt Simplicity
Authors
- Thibault Falque - Exakis Nelite
- Romain Wallon - CRIL, Univ Artois & CNRS
- Hugues Wattez - Laboratoire d'Informatique de l'X (LIX), École Polytechnique
About Metrics
Metrics is an open-source Python library developed at CRIL, designed to facilitate the conduction of experiments and their analysis.
The main objective of Metrics is to provide a complete toolchain from the execution of software programs to the analysis of their performance. In particular, the development of Metrics started with the observation that, in the SAT community, the process of experimenting solver remains mostly the same: everybody collects almost the same statistics about the solver execution. However, there are probably as many scripts as researchers in the domain for retrieving experimental data and drawing figures. There is thus clearly a need for a tool that unifies and makes easier the analysis of solver experiments.
The ambition of Metrics is thus to simplify the retrieval of experimental data from many different kinds of inputs (including the solver's output), and provide a nice interface for drawing commonly used plots, computing statistics about the execution of the solver, and effortlessly organizing them. In the end, the main purpose of Metrics is to favor the sharing and reproducibility of experimental results and their analysis.
Installation
To execute Metrics on your computer, you first need to install Python (at least version 3.8).
You may install Metrics using pip
, as the metrics
library is
available on PyPI.
pip install crillab-metrics
Note that, depending on your Python installation, you may need to use pip3
to install it, or to execute pip
as a module, as follows.
python3 -m pip install crillab-metrics
To improve the reproducibility of the experiments, we highly recommend to use
a virtual environment for
each analysis you create with Metrics, and thus to install the metrics
library in this virtual environment rather than with a system-wide
installation.
Using Metrics
You may find more information on how to use Metrics in the documentation we provide for the package.
Citing Metrics
If you are using Metrics in your papers, we kindly ask you to either refer to this repository or to one of the following papers:
- Metrics : Mission Expérimentations. Thibault Falque, Romain Wallon and Hugues Wattez. 16es Journées Francophones de Programmation par Contraintes (JFPC'21), 2021.
- Metrics: Towards a Unified Library for Experimenting Solvers. Thibault Falque, Romain Wallon and Hugues Wattez. 11th International Workshop on Pragmatics of SAT (POS'20), 2020.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file crillab_metrics-1.3.0.tar.gz
.
File metadata
- Download URL: crillab_metrics-1.3.0.tar.gz
- Upload date:
- Size: 2.5 MB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.1 CPython/3.10.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 84bbd601c0dd983ec83a62ce891ae2bb6270c354fde3352170c30015359b28bb |
|
MD5 | 96d6acf7bf4b99f59ead0aac7a2baf7e |
|
BLAKE2b-256 | 40ea354373b0f54f157123f225525323ec6b5692ee8730a807ef20b3a5475ad2 |
File details
Details for the file crillab_metrics-1.3.0-py3-none-any.whl
.
File metadata
- Download URL: crillab_metrics-1.3.0-py3-none-any.whl
- Upload date:
- Size: 103.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.1 CPython/3.10.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | a8fb9c3347f7f67c005da0c25839b989b81393d529d6cf76fb8e334fedd05014 |
|
MD5 | ad417388935deeccaa1bc454c66cfae2 |
|
BLAKE2b-256 | b440ea18731fef7790f5890af753ded993174cd62f10ce2f71de622bbdca4cf7 |