A Python library for working with A/B tests.
Project description
Ambrosia is a Python library for A/B tests design, split and effect measurement. It provides rich set of methods for conducting full A/B test pipeline.
An experiment design stage is performed using metrics historical data which could be processed in both forms of pandas and spark dataframes with either theoretical or empirical approach.
Group split methods support different strategies and multi-group split, which allows to quickly create control and test groups of interest.
Final effect measurement stage is conducted via testing tools that are able to return relative and absolute effects and construct corresponding confidence intervalsfor continious and binary variables. Testing tools as well as design ones support significant number of statistical criteria, like t-test, non-parametric, and bootstrap.
For additional A/B tests support library provides features and tools for data preproccesing and experiment acceleration.
Key functionality
Pilots design ✈
Multi-group split 🎳
Matching of new control group to the existing pilot 🎏
Getting the experiments result evaluation as p-value, point estimate of effect and confidence interval 🎞
Experiments acceleration 🎢
Documentation
For more details, see the Documentation and Tutorials.
Installation
Stable version is released on every tag to main branch.
pip install ambrosia
Ambrosia requires Python 3.7+
Usage
Designer
from ambrosia.designer import Designer
designer = Designer(dataframe=df, effects=1.2, metrics='portfel_clc') # 20% effect, and loaded data frame df
designer.run('size')
Splitter
from ambrosia.splitter import Splitter
splitter = Splitter(dataframe=df, id_column='id') # loaded data frame df with column with id - 'id'
splitter.run(groups_size=500, method='simple')
Tester
from ambrosia.tester import Tester
tester = Tester(dataframe=df, column_groups='group') # loaded data frame df with groups info 'group'
tester.run(metrics='retention', method='theory', criterion='ttest')
Development
To install all requirements run
make install
You must have python3 and poetry installed.
For autoformatting run
make autoformat
For linters check run
make lint
For tests run
make test
For coverage run
make coverage
To remove virtual environment run
make clean
Communication
Developers and evangelists:
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.