mETRICS - rEproducible sofTware peRformance analysIs in perfeCt Simplicity
Authors
- Thibault Falque - Exakis Nelite
- Romain Wallon - CRIL, Univ Artois & CNRS
- Hugues Wattez - Laboratoire d'Informatique de l'X (LIX), École Polytechnique
About Metrics
Metrics is an open-source Python library developed at CRIL, designed to facilitate the conduction of experiments and their analysis.
The main objective of Metrics is to provide a complete toolchain from the execution of software programs to the analysis of their performance. In particular, the development of Metrics started with the observation that, in the SAT community, the process of experimenting solver remains mostly the same: everybody collects almost the same statistics about the solver execution. However, there are probably as many scripts as researchers in the domain for retrieving experimental data and drawing figures. There is thus clearly a need for a tool that unifies and makes easier the analysis of solver experiments.
The ambition of Metrics is thus to simplify the retrieval of experimental data from many different kinds of inputs (including the solver's output), and provide a nice interface for drawing commonly used plots, computing statistics about the execution of the solver, and effortlessly organizing them. In the end, the main purpose of Metrics is to favor the sharing and reproducibility of experimental results and their analysis.
Installation
To execute Metrics on your computer, you first need to install Python (at least version 3.8).
You may install Metrics using pip, as the metrics library is
available on PyPI.
pip install crillab-metrics
Note that, depending on your Python installation, you may need to use pip3
to install it, or to execute pip as a module, as follows.
python3 -m pip install crillab-metrics
To improve the reproducibility of the experiments, we highly recommend to use
a virtual environment for
each analysis you create with Metrics, and thus to install the metrics
library in this virtual environment rather than with a system-wide
installation.
Using Metrics
You may find more information on how to use Metrics in the documentation we provide for the package.
Citing Metrics
If you are using Metrics in your papers, we kindly ask you to either refer to this repository or to one of the following papers:
- Metrics : Mission Expérimentations. Thibault Falque, Romain Wallon and Hugues Wattez. 16es Journées Francophones de Programmation par Contraintes (JFPC'21), 2021.
- Metrics: Towards a Unified Library for Experimenting Solvers. Thibault Falque, Romain Wallon and Hugues Wattez. 11th International Workshop on Pragmatics of SAT (POS'20), 2020.
2 maintainers
Release files for crillab-metrics 1.3.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| crillab_metrics-1.3.0.tar.gz | 2.5 MB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| crillab_metrics-1.3.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 2.6 MB
Release files / crillab_metrics-1.3.0.tar.gz
| Download URL | crillab_metrics-1.3.0.tar.gz |
|---|---|
| Size | 2.5 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
84bbd601c0dd983ec83a62ce891ae2bb6270c354fde3352170c30015359b28bb
|
|
BLAKE2b-256 checksum How to use checksums |
40ea354373b0f54f157123f225525323ec6b5692ee8730a807ef20b3a5475ad2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.10.6
|
Release files / crillab_metrics-1.3.0-py3-none-any.whl
| Download URL | crillab_metrics-1.3.0-py3-none-any.whl |
|---|---|
| Size | 103.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
a8fb9c3347f7f67c005da0c25839b989b81393d529d6cf76fb8e334fedd05014
|
|
BLAKE2b-256 checksum How to use checksums |
b440ea18731fef7790f5890af753ded993174cd62f10ce2f71de622bbdca4cf7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.10.6
|
