Skip to main content

rEproducible sofTware peRformance analysIs in perfeCt Simplicity

Project description

mETRICS - rEproducible sofTware peRformance analysIs in perfeCt Simplicity

License PyPI - Python Version PyPI - Status Travis (.org) Sonar Quality Gate Sonar Coverage

Authors

About Metrics

Metrics is an open-source Python library developed at CRIL, designed to facilitate the conduction of experiments and their analysis.

The main objective of Metrics is to provide a complete toolchain from the execution of software programs to the analysis of their performance. In particular, the development of Metrics started with the observation that, in the SAT community, the process of experimenting solver remains mostly the same: everybody collects almost the same statistics about the solver execution. However, there are probably as many scripts as researchers in the domain for retrieving experimental data and drawing figures. There is thus clearly a need for a tool that unifies and makes easier the analysis of solver experiments.

The ambition of Metrics is thus to simplify the retrieval of experimental data from many different kinds of inputs (including the solver's output), and provide a nice interface for drawing commonly used plots, computing statistics about the execution of the solver, and effortlessly organizing them. In the end, the main purpose of Metrics is to favor the sharing and reproducibility of experimental results and their analysis.

Installation

To execute Metrics on your computer, you first need to install Python (at least version 3.8).

You may install Metrics using pip, as the metrics library is available on PyPI.

pip install crillab-metrics

Note that, depending on your Python installation, you may need to use pip3 to install it, or to execute pip as a module, as follows.

python3 -m pip install crillab-metrics

To improve the reproducibility of the experiments, we highly recommend to use a virtual environment for each analysis you create with Metrics, and thus to install the metrics library in this virtual environment rather than with a system-wide installation.

Using Metrics

You may find more information on how to use Metrics in the documentation we provide for the package.

Citing Metrics

If you are using Metrics in your papers, we kindly ask you to either refer to this repository or to one of the following papers:

  • Metrics : Mission Expérimentations. Thibault Falque, Romain Wallon and Hugues Wattez. 16es Journées Francophones de Programmation par Contraintes (JFPC'21), 2021.
  • Metrics: Towards a Unified Library for Experimenting Solvers. Thibault Falque, Romain Wallon and Hugues Wattez. 11th International Workshop on Pragmatics of SAT (POS'20), 2020.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

crillab_metrics-1.3.0.tar.gz (2.5 MB view details)

Uploaded Source

Built Distribution

crillab_metrics-1.3.0-py3-none-any.whl (103.7 kB view details)

Uploaded Python 3

File details

Details for the file crillab_metrics-1.3.0.tar.gz.

File metadata

  • Download URL: crillab_metrics-1.3.0.tar.gz
  • Upload date:
  • Size: 2.5 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.10.6

File hashes

Hashes for crillab_metrics-1.3.0.tar.gz
Algorithm Hash digest
SHA256 84bbd601c0dd983ec83a62ce891ae2bb6270c354fde3352170c30015359b28bb
MD5 96d6acf7bf4b99f59ead0aac7a2baf7e
BLAKE2b-256 40ea354373b0f54f157123f225525323ec6b5692ee8730a807ef20b3a5475ad2

See more details on using hashes here.

File details

Details for the file crillab_metrics-1.3.0-py3-none-any.whl.

File metadata

File hashes

Hashes for crillab_metrics-1.3.0-py3-none-any.whl
Algorithm Hash digest
SHA256 a8fb9c3347f7f67c005da0c25839b989b81393d529d6cf76fb8e334fedd05014
MD5 ad417388935deeccaa1bc454c66cfae2
BLAKE2b-256 b440ea18731fef7790f5890af753ded993174cd62f10ce2f71de622bbdca4cf7

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page