Spotify Confidence
Python library for AB test analysis.
Why use Spotify Confidence?
Spotify Confidence provides convenience wrappers around statsmodel's various functions for computing p-values and confidence intervalls. With Spotify Confidence it's easy to compute several p-values and confidence bounds in one go, e.g. one for each country or for each date. Each function comes in two versions:
- one that return a pandas dataframe,
- one that returns a Chartify chart.
Spotify Confidence has support calculating p-values and confidence intervals using Z-statistics, Student's T-statistics (or more exactly Welch's T-test), as well as Chi-squared statistics. It also supports a variance reduction technique based on using pre-exposure data to fit a linear model.
There is also a Bayesian alternative in the BetaBinomial class.
Basic Example
import spotify_confidence as confidence
import pandas as pd
data = pd.DataFrame(
{'variation_name': ['treatment1', 'control', 'treatment2', 'treatment3'],
'success': [50, 40, 10, 20],
'total': [100, 100, 50, 60]
}
)
test = confidence.ZTest(
data,
numerator_column='success',
numerator_sum_squares_column=None,
denominator_column='total',
categorical_group_columns='variation_name',
correction_method='bonferroni')
test.summary()
test.difference(level_1='control', level_2='treatment1')
test.multiple_difference(level='control', level_as_reference=True)
test.summary_plot().show()
test.difference_plot(level_1='control', level_2='treatment1').show()
test.multiple_difference_plot(level='control', level_as_reference=True).show()
There's a lot more you can do:
- Segment results by one or more dimensions
- Use non-inferiority margins
- Group sequential tests
- Sample size and power calculations
- etc
See jupyter notebooks in examples folder for more complete examples.
Installation
Spotify Confidence can be installed via pip:
pip install spotify-confidence
Find the latest release version here
Code of Conduct
This project adheres to the Open Code of Conduct By participating, you are expected to honor this code.
3 maintainers
Release files for spotify-confidence 4.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| spotify_confidence-4.1.0.tar.gz | 66.4 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| spotify_confidence-4.1.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 156.8 kB
Release files / spotify_confidence-4.1.0.tar.gz
| Download URL | spotify_confidence-4.1.0.tar.gz |
|---|---|
| Size | 66.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
d71c6e617959c6c5c39de794347376344a611d74db686441d091698e1ae23e3f
|
|
BLAKE2b-256 checksum How to use checksums |
5f667bcc2f2d0e59977a10ef858d4a807b89ab8804013f5f13fab8d9be718203
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Release files / spotify_confidence-4.1.0-py3-none-any.whl
| Download URL | spotify_confidence-4.1.0-py3-none-any.whl |
|---|---|
| Size | 90.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
ae2dcbd8e6606ee03e25c91d62c7fc9c932e17d4e024183acbd5ce83d5bfc87c
|
|
BLAKE2b-256 checksum How to use checksums |
5e51987a6b554e51c4069489ab92d51f6febadf315838b934ac88dee269743e2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
