model-confidence-set provides a Python implementation of the Model Confidence Set (MCS) procedure (Hansen, Lunde, and Nason, 2011), a statistical method for comparing and selecting models based on their performance.
Project description
Model Confidence Set
The model-confidence-set
package provides a Python implementation of the Model Confidence Set (MCS) procedure (Hansen, Lunde, and Nason, 2011), a statistical method for comparing and selecting models based on their performance. It allows users to identify a set of models that are statistically indistinguishable from the best model, given a statistical confidence level.
This package
- supports both stationary and block bootstrap methods.
- implements two methods for p-value computation:
R
andSQ
. - optionally displays progress during computation.
Installation
To install model-confidence-set
, simply use pip:
pip install model-confidence-set
Usage
To use the Model Confidence Set in your Python code, follow the example below:
import numpy as np
import pandas as pd
from model_confidence_set import ModelConfidenceSet
# Example losses matrix where rows are observations and columns are models
losses = np.random.rand(100, 5) # 100 observations for 5 models
# Initialize the MCS procedure (5'000 bootstrap iterations, 5% confidence level)
mcs = ModelConfidenceSet(losses, n_boot=5000, alpha=0.05, show_progress=True)
# Compute the MCS
mcs.compute()
# Retrieve the results as a pandas DataFrame (use as_dataframe=False for a dict)
results = mcs.results()
print(results)
Parameters
losses
: A 2Dnumpy.ndarray
orpandas.DataFrame
containing loss values of models. Rows correspond to observations, and columns correspond to different models.n_boot
: Number of bootstrap replications for computing p-values. Default is5000
.alpha
: Significance level for determining model confidence set. Default is0.05
.block_len
: The length of blocks for the block bootstrap. IfNone
, it defaults to the number of observations.bootstrap_variant
: Specifies the bootstrap variant to use. Options are'stationary'
or'block'
. Default is'stationary'
.method
: The method used for p-value calculation. Options are'R'
or'SQ'
. Default is'R'
.show_progress
: Whether to show a progress bar during bootstrap computations. Default isFalse
.
Acknowledgments
This package follows the Matlab implementation by Kevin Sheppard and draws inspiration from the Python implementation by Michael Gong.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for model-confidence-set-0.1.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | cd52d2e9bb7b819207b53c8f934d4daeb8fa92a5d551ae350b3e1c3254458b7f |
|
MD5 | 0fa0cf353eb6c207776de1ce8b29f8bb |
|
BLAKE2b-256 | 200d5df5aad24b38acf7185c3e2b5232d2c53200d40a9cb7c2c3bdb99eec7249 |
Hashes for model_confidence_set-0.1.0-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9f172badb7f76d89ec4fc2c1215fa276c214576490193f7b67efab0ced9335f0 |
|
MD5 | 9c7a7418638d5cb5499bf97f8f5394b5 |
|
BLAKE2b-256 | 635ef57d5889c92fa4f857b1891aa316863305cb21a5f3d1925c14b305dfacdd |