Ambrosia 0.2.0

A Python library for working with A/B tests.

Ambrosia is a Python library for A/B tests design, split and effect measurement. It provides rich set of methods for conducting full A/B test pipeline.

An experiment design stage is performed using metrics historical data which could be processed in both forms of pandas and spark dataframes with either theoretical or empirical approach.

Group split methods support different strategies and multi-group split, which allows to quickly create control and test groups of interest.

Final effect measurement stage is conducted via testing tools that are able to return relative and absolute effects and construct corresponding confidence intervalsfor continious and binary variables. Testing tools as well as design ones support significant number of statistical criteria, like t-test, non-parametric, and bootstrap.

For additional A/B tests support library provides features and tools for data preproccesing and experiment acceleration.

Key functionality

• Pilots design ✈

• Multi-group split 🎳

• Matching of new control group to the existing pilot 🎏

• Getting the experiments result evaluation as p-value, point estimate of effect and confidence interval 🎞

• Experiments acceleration 🎢

Documentation

For more details, see the Documentation and Tutorials.

Installation

Stable version is released on every tag to main branch.

pip install ambrosia

Ambrosia requires Python 3.7+

Usage

Designer

from ambrosia.designer import Designer
designer = Designer(dataframe=df, effects=1.2, metrics='portfel_clc') # 20% effect, and loaded data frame df
designer.run('size')

Splitter

from ambrosia.splitter import Splitter
splitter = Splitter(dataframe=df, id_column='id') # loaded data frame df with column with id - 'id'
splitter.run(groups_size=500, method='simple')

Tester

from ambrosia.tester import Tester
tester = Tester(dataframe=df, column_groups='group') # loaded data frame df with groups info 'group'
tester.run(metrics='retention', method='theory', criterion='ttest')

Development

To install all requirements run

make install

You must have python3 and poetry installed.

For autoformatting run

make autoformat

For linters check run

make lint

For tests run

make test

For coverage run

make coverage

To remove virtual environment run

make clean

Communication

Developers and evangelists:

• Bayramkulov Aslan

• Khakimov Artem

• Vasin Artem