Skip to main content

A tiny framework to perform adversarial validation of your training and test data.

Project description

adversarial-validation

PyPI PyPI - Python Version codecov Code style: black GitHub Workflow Status Documentation Status PyPI - Wheel

A tiny framework to perform adversarial validation of your training and test data.

What is adversarial validation?

A common workflow in machine learning projects (especially in Kaggle competitions) is to:

  1. train your ML model in a training dataset.
  2. tune and validate your ML model in a validation dataset (which typically originates as a fraction of the training dataset).
  3. finally, assess the actual generalization ability of your ML model in a held-out test dataset.

This strategy is widely accepted, but it heavily relies on the assumption that the training and test datasets are drawn from the same underlying distribution. This is often referred to as the “identically distributed” property in the literature.

This package helps you easily assert whether the "identically distributed" property holds true for your training and test datasets or equivalently whether your validation dataset is a good proxy for your model's performance on the unseen test instances.

Install

The recommended installation is via pip:

pip install advertion

(advertion stands for adversarial validation)

Usage

from advertion import validate

train = pd.read_csv("...")
test = pd.read_csv("...")

validate(
    trainset=train,
    testset=test,
    target="label",
)

# // {
# //     "datasets_follow_same_distribution": True,
# //     'mean_roc_auc': 0.5021320833333334,
# //     "adversarial_features': ['id'],
# // }

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

advertion-0.1.0b0.tar.gz (5.4 kB view hashes)

Uploaded Source

Built Distribution

advertion-0.1.0b0-py3-none-any.whl (6.2 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page