Skip to main content

Package to calibrate and understand ML Models

Project description

ML Insights

Welcome to ML-Insights!

This package contains two core sets of functions:

  1. Calibration
  2. Interpreting Models

For probability calibration, the main class is SplineCalib. Given a set of model outputs and the “true” classes, you can fit a SplineCalib object. That object can then be used to calibrate future model predictions post-hoc.

>>> model.fit(X_train, y_train)
>>> sc = mli.SplineCalib()
>>> sc.fit(X_valid, y_valid)
>>> uncalib_preds = model.predict_proba(X_test)
>>> calib_preds = sc.calibrate(uncalib_preds)
>>> cv_preds = mli.cv_predictions(model, X_train, y_train)
>>> model.fit(X_train, y_train)
>>> sc = mli.SplineCalib()
>>> sc.fit(cv_preds, y_train)
>>> uncalib_preds = model.predict_proba(X_test)
>>> calib_preds = sc.calibrate(uncalib_preds)

For model interpretability, we provide the ice_plot and histogram_pair functions as well as other tools.

>>> rd = mli.get_range_dict(X_train)
>>> mli.ice_plot(model, X_test.sample(3), X_train.columns, rd)
>>> mli.histogram_pair(df.outcome, df.feature, bins=np.linspace(0,100,11))

Please see the documentation and examples at the links below.

Python

Python 3.4+

Disclaimer

We have tested this tool to the best of our ability, but understand that it may have bugs. It was most recently developed on Python 3.7.3. Use at your own risk, but feel free to report any bugs to our github. <https://github.com/numeristical/introspective>

Installation

$ pip install ml_insights

Usage

>>> import ml_insights as mli
>>> xray = mli.ModelXRay(model, data)
>>> rfm = RandomForestClassifier(n_estimators = 500, class_weight='balanced_subsample')
>>> rfm_cv = mli.SplineCalibratedClassifierCV(rfm)
>>> rfm_cv.fit(X_train,y_train)
>>> test_res_calib_cv = rfm_cv.predict_proba(X_test)[:,1]
>>> log_loss(y_test,test_res_calib_cv)

Source

Find the latest version on github: https://github.com/numeristical/introspective

Feel free to fork and contribute!

License

Free software: MIT license

Developed By

  • Brian Lucena
  • Ramesh Sampath

References

Alex Goldstein, Adam Kapelner, Justin Bleich, and Emil Pitkin. 2014. Peeking Inside the Black Box: Visualizing Statistical Learning With Plots of Individual Conditional Expectation. Journal of Computational and Graphical Statistics (March 2014)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for ml-insights, version 0.1.6
Filename, size File type Python version Upload date Hashes
Filename, size ml_insights-0.1.6-py2.py3-none-any.whl (39.2 kB) File type Wheel Python version py2.py3 Upload date Hashes View
Filename, size ml_insights-0.1.6.tar.gz (32.1 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page