No project description provided
Project description
confidence_interval_estimator_ML
This repo contains code to perform estimation of Confidence Intervals both parametric and non-parametric (a.k.a. Monte Carlo Confidence Interval or Empirical Confidence Interval estimation) for Machine Learing models.
Colab Demo
You can easily try this tool in Google Colab by clicking on the following badge:
Quickstart
>>> import confidence_interval_estimator_ML.utils as cie
>>> # Parametric Estimation:
>>> cie.parametric_confidence_interval(confidence_level=.95, mean=model_accuracy, sample_size=sample_size, verbose=True)
Computing C.I. assuming the data follow a standard normal distribution
With 95.0% confidence the accuracy of the model is in 0.977 +/-0.023.
(0.9773132867658845, 0.02268671323411553)
>>> # Non-Parametric Estimation:
>>> test_accuracies = cie.get_accuracy_on_samples(get_classifier_instance, X, y, n_iterations=100, sample_ratio=1,
train_ratio=0.8, random_seed=None, sample_with_replacement=True, verbose=False)
>>> cie.nonparametric_confidence_interval(accs, .9)
From the given data, with 90.0% confidence the accuracy of the model is in 0.959 +/-4.05.
(0.9595, 0.04050000000000009)
Further Details
You can find further details on the usage of the tool in the code documentation or in the examples provided in the Notebook.ipynb file. If you need help, please don't hesitate to open a github issue or getting in touch.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for confidence_interval_estimator_ML-0.1.2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | b7764a5e4c9eaf7278bfe3dfe7820378c3db8cec7fa15fe470e578fd9d8ff2e3 |
|
MD5 | 9a039f18d884f9a063ef4efca374981a |
|
BLAKE2b-256 | 6e2a83843fdf4237f147648310aac7b22378979eb89a6ef859e453fe521cc885 |
Close
Hashes for confidence_interval_estimator_ML-0.1.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4e2596e3d84dcb8416fad883b1a96c22530e0464f484f51ec28ff1a79dc7e887 |
|
MD5 | 749356f1964cd8e4bdad6c67551ff1a1 |
|
BLAKE2b-256 | 1ab196178263058f76868bc865b746c44b7d3ea88fd965a46899dcf95482fe3d |