Skip to main content

A/B experiments planning and evaluation tool

Project description

Experiment report

PyPI Latest Release PyPI Downloads License - MIT License

ABacus: fast hypothesis testing and experiment design solution

ABacus is a Python library developed for A/B experimentation and testing. It includes versatile instruments for different experimentation tasks like prepilot, sample size determination, results calculation, visualisations and reporting.

Important features

  • Experiment design: type I and II errors, effect size, sample size simulations.
  • Groups splitting with flexible configuration and stratification.
  • A/A test and evaluation of splitter accuracy.
  • Evaluation of experiment results with various statistical tests and approaches.
  • Sensitivity increasing techniques like stratification, CUPED and CUPAC.
  • Visualisation of experiment.
  • Reporting in a human-readable format.

Installation

You can use pip to install ABacus directly from PyPI:

pip install kolmogorov-abacus

or right from GitHub:

pip install pip+https://github.com/kolmogorov-lab/abacus

Note the requirement of Python 3.11+.

Quick example

To define an experiment and analyse it is as easy as to describe your experiment and data:

from abacus.auto_ab.abtest import ABTest
from abacus.auto_ab.params import ABTestParams, DataParams, HypothesisParams

data_params = DataParams(...)
hypothesis_params = HypothesisParams(...)
ab_params = ABTestParams(data_params, hypothesis_params)

data = pd.read_csv('abtest_data.csv')

ab_test = ABTest(data, ab_params)

ab_test.report()

The result of code execution is the following:

Experiment report

Documentation and Examples

Detailed documentation and examples are available for your usage.

Communication

Authors and developers:

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

kolmogorov-abacus-0.0.4.tar.gz (30.8 kB view hashes)

Uploaded Source

Built Distribution

kolmogorov_abacus-0.0.4-py3-none-any.whl (34.5 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page