Y-Scramble: a package for y-randomization validation
Project description
Y-Scramble
Y-Scramble is a simple python package to perform y-randomization validation
of machine learning models. It can be used for classification and regression tasks
and accepts models following the scikit-learn
inteface, and the user may use all scorers available at scikit-learn
(accuracy
, recall
, precision
).
Installing
Y-Scramble can be installed from PyPI using the following command:
$ pip install y-scamble
Usage
from y_scramble import Scrambler
from sklearn.tree import DecisionTreeClassifier
X, y = load_iris(return_X_y=True)
model = DecisionTreeClassifier()
scrambler = Scrambler(model=model, iterations=1000)
scores, zscores, pvalues, significances = scrambler.validate(
X, y,
scoring="accuracy",
cross_val_score_aggregator="mean",
pvalue_threshold=0.01
)
The scramble
object returns the scores, z-scores, p-values and the significancy
information for the model trained (base_model
) using the default dataset and for different randomized versions as well (scrambled_models
). These results are stores in numpy arrays, where the position of index 0 represents the base_model
and the others the scrambled_models
.
The score of the base_model
is stored in scores[0]
, and i's p-values is stored in
pvalues[0]
. If this p-value is significant, the value of significances[0]
will be
True
, indicating that base_model
shows a significantly better result when comparing to the randomized models. Following the same logic, scores[1]
to scores[1000]
, for example, will store the score values for the randomized model 1
and 1000
, respectively.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
Hashes for y_scramble-0.0.4-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 015a16bc8e17f37655b98920feae21af9b995d21c08b896f709040cf0fbad157 |
|
MD5 | 9091055965c8d720132d15cab8dce03d |
|
BLAKE2b-256 | 9ac650973da35822f35fd2273ab5c5e06feefe47e8d0ad5120d8301b04dad20f |