model-confidence-set provides a Python implementation of the Model Confidence Set (MCS) procedure (Hansen, Lunde, and Nason, 2011), a statistical method for comparing and selecting models based on their performance.
Project description
Model Confidence Set
The model-confidence-set
package provides a Python implementation of the Model Confidence Set (MCS) procedure (Hansen, Lunde, and Nason, 2011), a statistical method for comparing and selecting models based on their performance. It allows users to identify a set of models that are statistically indistinguishable from the best model, given a statistical confidence level.
This package
- supports both stationary and block bootstrap methods.
- implements two methods for p-value computation: relative and sequential.
- optionally displays progress during computation.
Installation
To install model-confidence-set
, simply use pip:
pip install model-confidence-set
Usage
To use the Model Confidence Set in your Python code, follow the example below:
import numpy as np
import pandas as pd
from model_confidence_set import ModelConfidenceSet
# Example losses matrix where rows are observations and columns are models
losses = np.random.rand(100, 5) # 100 observations for 5 models
# Initialize the MCS procedure (5'000 bootstrap iterations, 5% confidence level)
mcs = ModelConfidenceSet(losses, n_boot=5000, alpha=0.05, show_progress=True)
# Compute the MCS
mcs.compute()
# Retrieve the results as a pandas DataFrame (use as_dataframe=False for a dict)
results = mcs.results()
print(results)
Parameters
losses
: A 2Dnumpy.ndarray
orpandas.DataFrame
containing loss values of models. Rows correspond to observations, and columns correspond to different models.n_boot
: Number of bootstrap replications for computing p-values. Default is5000
.alpha
: Significance level for determining model confidence set. Default is0.05
.block_len
: The length of blocks for the block bootstrap. IfNone
, it defaults to the number of observations.bootstrap_variant
: Specifies the bootstrap variant to use. Options are'stationary'
or'block'
. Default is'stationary'
.method
: The method used for p-value calculation. Options are'R'
for relative or'SQ'
for sequential. Default is'R'
.show_progress
: Whether to show a progress bar during bootstrap computations. Default isFalse
.
Acknowledgments
This package draws inspiration from
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for model-confidence-set-0.1.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2b5df8d7a83cf5d315ee63c6047ddc284017eaba245c44db1c01d56ede90b087 |
|
MD5 | 1abf233114f5a98b66ed9493a57a6c6e |
|
BLAKE2b-256 | cf0e839d012dde2c931393053e4ba6a0ad9f917a5386d6815ca112c5f25cea0a |
Hashes for model_confidence_set-0.1.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 677f9e05d9dacd6eba9165db3d92dca6119be71e478ad920232809e7952b5d11 |
|
MD5 | 3af4c28bb623cdb0fef3851190be4b5b |
|
BLAKE2b-256 | a8ac196de9a41276bf1cbe7644f673657324930f9d87ccb401252a0425e7e6f6 |