scikit-fallback 0.0.1

Machine learning with a reject option

scikit-fallback is a scikit-learn-compatible Python package for machine learning with a reject option.

🏗 Installation

scikit-fallback requires:

• Python (>=3.9,< 3.13)
• scikit-learn (>=1.3)
• matplotlib (>=3.0) (optional)

pip install -U scikit-fallback

👩‍💻 Usage

To allow your model to fallback—i.e., abstain from predictions—you can wrap your classification pipeline with a scikit-fallback rejector and then train the final pipeline and evaluate both the classifier's and the rejector's performance.

For example, RateFallbackClassifierCV fits the base estimator and then finds the best confidence threshold s.t. the fallback rate on the held-out set is <= the provided value. If fallback_mode == "store", then the rejector returns FBNDArrays of predictions and a sparse fallback-mask property, which lets us summarize the accuracy of both predictions and rejections.

from skfb.estimators import RateFallbackClassifierCV
from sklearn.linear_model import LogisticRegressionCV

rejector = RateFallbackClassifierCV(
    LogisticRegressionCV(cv=4, random_state=0),
    fallback_rates=(0.05, 0.06, 0.07),
    cv=5,
    fallback_label=-1,
    fallback_mode="store",
)
rejector.fit(X_train, y_train)
rejector.score(X_test, y_test)

For more information, see the project's Wiki.

📚 Examples

See the examples/ directory for various applications of fallback estimators and scorers to scikit-learn-compatible pipelines.

🔗 References

1. Hendrickx, K., Perini, L., Van der Plas, D. et al. Machine learning with a reject option: a survey. Mach Learn 113, 3073–3110 (2024). https://doi.org/10.1007/s10994-024-06534-x