Skip to main content

Y-Scramble: a package for y-randomization validation

Project description

Y-Scramble

Y-Scramble is a simple python package to perform y-randomization validation of machine learning models. It can be used for classification and regression tasks and accepts models following the scikit-learninteface, and the user may use all scorers available at scikit-learn (accuracy, recall, precision).

Installing

Y-Scramble can be installed from PyPI using the following command:

$ pip install y-scamble

Usage

from y_scramble import Scrambler
from sklearn.tree import DecisionTreeClassifier

X, y = load_iris(return_X_y=True)
model = DecisionTreeClassifier()
scrambler = Scrambler(model=model, iterations=1000)

scores, zscores, pvalues, significances = scrambler.validate(
    X, y, 
    scoring="accuracy", 
    cross_val_score_aggregator="mean", 
    pvalue_threshold=0.01
)

The scramble object returns the scores, z-scores, p-values and the significancy information for the model trained (base_model) using the default dataset and for different randomized versions as well (scrambled_models). These results are stores in numpy arrays, where the position of index 0 represents the base_modeland the others the scrambled_models.

The score of the base_model is stored in scores[0], and i's p-values is stored in pvalues[0]. If this p-value is significant, the value of significances[0] will be True, indicating that base_model shows a significantly better result when comparing to the randomized models. Following the same logic, scores[1] to scores[1000], for example, will store the score values for the randomized model 1 and 1000, respectively.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

y_scramble-0.0.8-py3-none-any.whl (3.7 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page