Skip to main content

Y-Scramble: a package for y-randomization validation

Project description

Y-Scramble

Y-Scramble is a simple python package to perform y-randomization validation of machine learning models. It can be used for classification and regression tasks and accepts models following the scikit-learninteface, and the user may use all scorers available at scikit-learn (accuracy, recall, precision).

Installing

Y-Scramble can be installed from PyPI using the following command:

$ pip install y-scamble

Usage

from y_scramble import Scrambler
from sklearn.tree import DecisionTreeClassifier

X, y = load_iris(return_X_y=True)
model = DecisionTreeClassifier()
scrambler = Scrambler(model=model, iterations=1000)

scores, zscores, pvalues, significances = scrambler.validate(
    X, y, 
    scoring="accuracy", 
    cross_val_score_aggregator="mean", 
    pvalue_threshold=0.01
)

The scramble object returns the scores, z-scores, p-values and the significancy information for the model trained (base_model) using the default dataset and for different randomized versions as well (scrambled_models). These results are stores in numpy arrays, where the position of index 0 represents the base_modeland the others the scrambled_models.

The score of the base_model is stored in scores[0], and i's p-values is stored in pvalues[0]. If this p-value is significant, the value of significances[0] will be True, indicating that base_model shows a significantly better result when comparing to the randomized models. Following the same logic, scores[1] to scores[1000], for example, will store the score values for the randomized model 1 and 1000, respectively.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

y_scramble-0.0.8-py3-none-any.whl (3.7 kB view details)

Uploaded Python 3

File details

Details for the file y_scramble-0.0.8-py3-none-any.whl.

File metadata

  • Download URL: y_scramble-0.0.8-py3-none-any.whl
  • Upload date:
  • Size: 3.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/3.10.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.8.12

File hashes

Hashes for y_scramble-0.0.8-py3-none-any.whl
Algorithm Hash digest
SHA256 048a7a59a44f21b3481bef7ad8faf876d5b4f2eeea002eab1271accb54adb436
MD5 94b82a5a2c6e60f368fdb983cf0a3419
BLAKE2b-256 c5c9e826d8fec2377d2825dc09815b9feb13bb94dff4385a1b5ac74c4c380113

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page