abby

A/B testing for Human

Abby is a A/B testing library package for human. Abby aims to make A/B testing as easy as ABC and accessible to anyone.

Installation

$ pip install abby

Quick Start

Please note that your variant name column should be named as variant_name. Otherwise, the method will raise an error and ask you to adjust the column name accordingly.

A/B testing for continuous metric

from abby.datasets import load_dataset
from abby import compare_ttest

data = load_dataset('click_impression')

compare_ttest(data=data, variants=['control', 'experiment'], numerator='click')

A/B testing for ratio metric

from abby.datasets import load_dataset
from abby import compare_delta

data = load_dataset("click_impression")

compare_delta(
    data=data,
    variants=["control", "experiment"],
    numerator="click",
    denominator="impression",
)

A/B testing for ratio metric using bootstrap

from abby.datasets import load_dataset
from abby import compare_bootstrap_delta

data = load_dataset("click_impression")

compare_bootstrap_delta(
    data=data,
    variants=["control", "experiment"],
    numerator="click",
    denominator="impression",
    n_bootstrap=10_000,
)

A/B testing for multiple metrics

from abby.datasets import load_dataset
from abby import compare_multiple, Ratio

data = load_dataset("click_impression")

result = compare_multiple(
    data=data,
    variants=["control", "experiment"],
    metrics=["click", Ratio("click", "impression")],
)

A/B/N testing for multiple metrics

from abby.datasets import load_dataset
from abby import compare_multiple, Ratio

data = load_dataset("click_impression_3_variants")

result = compare_multiple(
    data=data,
    variants=["control", "experiment_A", "experiment_B"],
    metrics=["click", Ratio("click", "impression")],
)