ldr 0.3

Latent Dimensionality Reduction in Python

LDR

---

LDR stands for Latent Dimensionality Reduction. It is a generic method for interpreting models. It is deployed here as a python module.

About

The purpose of LDR is to solve a common and controversial problem. Often models that have a higher predictive accuracy are more complex. These complex models are sensibly referred to as black box models. This is frustrating for many data scientists, as they end up with a model that performs well, but they can't explain why, which can cause the model to fail in critical situations which are difficult to test for.

LDR aims to bridge that gap by providing a generic, reliable algorithmic method for interpreting most models. I define interpretability as:

1. Understanding the quality of a systems current understanding. Is the model overfitting, is there insufficient training data, and thus will it fail when deployed to the real world?

2. Interpreting how the value of a feature, or subset of features, affects a model's prediction (which I refer to here as feature interpretation).

3. The ability to use a model when not all values for the input features are present.

Getting Started

Prerequesites

Python3.

Installation

python3 -m pip install ldr --user

Usage

An example analysis of a simple generated distribution can be found here.

An example analysis of a classification problem can be found here.

An example analysis of a regression problem can be found here.

A step by step multi-model interpolation example can be found, where outlier detection is enforced to improve the efficacy for critical systems here.

Additional Notes

If you find this package useful, please consider contributing!