Y-Scramble: a package for y-randomization validation
Project description
Y-Scramble
Y-Scramble is a simple python package to perform y-randomization validation
of machine learning models. It can be used for classification and regression tasks
and accepts models following the scikit-learn
inteface, and the user may use all scorers available at scikit-learn
(accuracy
, recall
, precision
).
Installing
Y-Scramble can be installed from PyPI using the following command:
$ pip install y-scamble
Usage
from y_scramble import Scrambler
from sklearn.tree import DecisionTreeClassifier
X, y = load_iris(return_X_y=True)
model = DecisionTreeClassifier()
scrambler = Scrambler(model=model, iterations=1000)
scores, zscores, pvalues, significances = scrambler.validate(
X, y,
scoring="accuracy",
cross_val_score_aggregator="mean",
pvalue_threshold=0.01
)
The scramble
object returns the scores, z-scores, p-values and the significancy
information for the model trained (base_model
) using the default dataset and for different randomized versions as well (scrambled_models
). These results are stores in numpy arrays, where the position of index 0 represents the base_model
and the others the scrambled_models
.
The score of the base_model
is stored in scores[0]
, and i's p-values is stored in
pvalues[0]
. If this p-value is significant, the value of significances[0]
will be
True
, indicating that base_model
shows a significantly better result when comparing to the randomized models. Following the same logic, scores[1]
to scores[1000]
, for example, will store the score values for the randomized model 1
and 1000
, respectively.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
Hashes for y_scramble-0.0.8-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 048a7a59a44f21b3481bef7ad8faf876d5b4f2eeea002eab1271accb54adb436 |
|
MD5 | 94b82a5a2c6e60f368fdb983cf0a3419 |
|
BLAKE2b-256 | c5c9e826d8fec2377d2825dc09815b9feb13bb94dff4385a1b5ac74c4c380113 |