MAPIE 0.1.3

A scikit-learn-compatible module for estimating prediction intervals.

MAPIE - Model Agnostic Prediction Interval Estimator

MAPIE allows you to easily estimate prediction intervals using your favourite sklearn-compatible regressor.

🛠 Installation

Install via pip:

pip install mapie

To install directly from the github repository :

pip install git+https://github.com/simai-ml/MAPIE

⚡️ Quickstart

Let us start with a basic regression problem. Here, we generate one-dimensional noisy data that we fit with a linear model.

import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.datasets import make_regression

regressor = LinearRegression()
X, y = make_regression(n_samples=500, n_features=1, noise=20, random_state=59)

Since MAPIE is compliant with the standard scikit-learn API, we follow the standard sequential fit and predict process like any scikit-learn regressor.

from mapie.estimators import MapieRegressor
mapie = MapieRegressor(regressor, method="jackknife_plus")
mapie.fit(X, y)
y_preds = mapie.predict(X)

MAPIE returns a np.ndarray of shape (n_samples, 3) giving the predictions, as well as the lower and upper bounds of the prediction intervals for the target quantile. The estimated prediction intervals can then be plotted as follows.

from matplotlib import pyplot as plt
from mapie.metrics import coverage_score
plt.xlabel('x')
plt.ylabel('y')
plt.scatter(X, y, alpha=0.3)
plt.plot(X, y_preds[:, 0], color='C1')
order = np.argsort(X[:, 0])
plt.fill_between(X[order].ravel(), y_preds[:, 1][order], y_preds[:, 2][order], alpha=0.3)
plt.title(
    f"Target coverage = 0.9; Effective coverage = {coverage_score(y, y_preds[:, 1], y_preds[:, 2])}"
)
plt.show()

The title of the plot compares the target coverage with the effective coverage. The target coverage, or the confidence interval, is the fraction of true labels lying in the prediction intervals that we aim to obtain for a given dataset. It is given by the alpha parameter defined in MapieRegressor, here equal to the default value of 0.1 thus giving a target coverage of 0.9. The effective coverage is the actual fraction of true labels lying in the prediction intervals.

📘 Documentation

The documentation can be found on this link. It contains the following sections:

• Quickstart

• Theoretical description

• Tutorial

• API

• Examples

📝 Contributing

You are welcome to propose and contribute new ideas. We encourage you to open an issue so that we can align on the work to be done. It is generally a good idea to have a quick discussion before opening a pull request that is potentially out-of-scope. For more information on the contribution process, please go here.

🤝 Affiliations

MAPIE has been developed through a collaboration between Quantmetry, Michelin, and ENS Paris-Saclay with the financial support from Région Ile de France.

💬 Citations

MAPIE methods are based on the work by Foygel-Barber et al. (2020).

Rina Foygel Barber, Emmanuel J. Candès, Aaditya Ramdas, and Ryan J. Tibshirani. Predictive inference with the jackknife+. Ann. Statist., 49(1):486–507, 022021

📝 License

MAPIE is free and open-source software licensed under the 3-clause BSD license.