ablisk

This package aims to make it easier for data scientists, analysts, and engineers to conduct statistically sound experiments. At its core is the ABLisk class. This class provides tools for designing, analyzing, and simulating A/B tests.

Features

• Sample Size Calculation: Calculate the minimum required sample size.
• Result Analysis: Retrieve and visualize experiment results with support for confidence intervals and kernel density estimates (KDEs).
• Recommendation: Generate results summary and recommendations based os results.

Class Overview

ABLisk

The main class provides the following methods and attributes:

Initialization

ABLisk(bcr, mde, alpha = 0.05, power = 0.8, is_absolute_variation: bool = True, is_two_tailed: bool = True)

• bcr: Baseline Conversion Rate (0 <= mde < 1).
• mde: Minimum Detectable Effect (absolute or relative) (0 < mde < 1).
• alpha: Significance level (default: 0.05).
• power: Statistical power (default: 0.8).
• is_absolute_variation: Whether mde is absolute (default: True).
• ** is_two_tailed**: Use two-tailed tests (default: True).

Methods

1. get_sample_size()

   • Calculates the minimum required sample size.

2. get_experiment_results(n_ctrl, p_ctrl, n_trmt, p_trmt, plot, full_summary, from_dataset, dataset)

   • Analyzes and visualizes results.
   • Parameters:
     • n_ctrl, n_trmt: Sample sizes of the control and treatment groups.
     • p_ctrl, p_trmt: Conversion rates for control and treatment groups.
     • plot: for visualization('KDE' or 'Confidence Intervals') or results summary and recommendations (None).
     • full_summary: whether to pass the results summary as a string (to be printed) or as a tuple containing pieces of this same summary.
     • from_dataset: whether to get sample size and proportions from an experiment dataset or bay manually inserting in their respective arguments.
     • dataset: the csv file path or DataFrame with three arbitrary named columns describing the user ID, the variants ("treatment" or "control"), and whether the user as converted ({"yes", "no"} case-insensitive, 1, 0)

Usage

Installation

pip install ablisk

Importing the module

from ablisk.core import ABLisk

Example: Calculate Sample Size

ab_test = ABLisk(bcr = 0.1, mde = 0.02, alpha = 0.05, power = 0.8)
sample_size = ab_test.get_sample_size()
print(f"Required Sample Size: {sample_size}")

Example: Visualize Experiment Results

ab_test.get_experiment_results(n_ctrl = 500, p_ctrl = 0.1, n_trmt = 500, p_trmt = 0.12, plot_type = 'Error Bars')

💡 A more detailed example regarding the implementation of this tool is available on its respective notebook. You can view the notebook using nbviewer due to GitHub rendering limitations here.

License

This project is licensed under the MIT License. See the LICENSE file for details.

Contribution

Contributions are welcome! Please fork the repository, create a feature branch, and submit a pull request.

References

• FÁVERO, L. P.; BELFIORE, P. Manual de Análise de Dados: estatística e modelagem multivariada com Excel®, SPSS® e Stata®. Rio de Janeiro: Elsevier, 2017.
• GRAVETTER, F. J.; WALLNAU, L. B. Statistics for the Behavioral Sciences. 10th ed. Boston: Cengage Learning, 2015.
• SAINANI K. Stanford University. Introduction to Sample Size and Power Calculations. Last accessed on Dec 28 2024.
• UDACITY. A/B Testing. Last accessed on Dec 28 2024.

Acknowledgments

This project would not be possible without the massive contribution of Evan Miller regarding A/B testing methodologies and tools. Refer to his A/B Testing Sample Size Calculator for further details.

Explore the Web App 🌐🚀

ablisk is the statistical engine of xplendid – the AI-assisted web application version for this module.

Networking

Connect with me:

/domingosdeeulariadumba

/domingosdeeulariadumba

/domingosdeeulariadumba