Skip to main content

Package for calculating and visualising confidence intervals, e.g. for A/B test analysis.

Project description

Spotify Confidence

Status Latest release Python Python Python

Python library for AB test analysis.

Why use Spotify Confidence?

Spotify Confidence provides convenience wrappers around statsmodel's various functions for computing p-values and confidence intervalls. With Spotify Confidence it's easy to compute several p-values and confidence bounds in one go, e.g. one for each country or for each date. Each function comes in two versions:

  • one that return a pandas dataframe,
  • one that returns a Chartify chart.

Spotify Confidence has support calculating p-values and confidence intervals using Z-statistics, Student's T-statistics (or more exactly Welch's T-test), as well as Chi-squared statistics. It also supports a variance reduction technique based on using pre-exposure data to fit a linear model.

There is also a Bayesian alternative in the BetaBinomial class.

Basic Example

import spotify_confidence as confidence
import pandas as pd

data = pd.DataFrame(
    {'variation_name': ['treatment1', 'control', 'treatment2', 'treatment3'],
     'success': [50, 40, 10, 20],
     'total': [100, 100, 50, 60]
    }
)

test = confidence.ZTest(
    data,
    numerator_column='success',
    numerator_sum_squares_column=None,
    denominator_column='total',
    categorical_group_columns='variation_name',
    correction_method='bonferroni')
    
test.summary()
test.difference(level_1='control', level_2='treatment1')
test.multiple_difference(level='control', level_as_reference=True)

test.summary_plot().show()
test.difference_plot(level_1='control', level_2='treatment1').show()
test.multiple_difference_plot(level='control', level_as_reference=True).show()

There's a lot more you can do:

  • Segment results by one or more dimensions
  • Use non-inferiority margins
  • Group sequential tests
  • Sample size and power calculations
  • etc

See jupyter notebooks in examples folder for more complete examples.

Installation

Spotify Confidence can be installed via pip:

pip install spotify-confidence

Find the latest release version here

Code of Conduct

This project adheres to the Open Code of Conduct By participating, you are expected to honor this code.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

spotify_confidence-4.0.0.tar.gz (96.7 kB view details)

Uploaded Source

Built Distribution

spotify_confidence-4.0.0-py3-none-any.whl (86.6 kB view details)

Uploaded Python 3

File details

Details for the file spotify_confidence-4.0.0.tar.gz.

File metadata

  • Download URL: spotify_confidence-4.0.0.tar.gz
  • Upload date:
  • Size: 96.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.12.7

File hashes

Hashes for spotify_confidence-4.0.0.tar.gz
Algorithm Hash digest
SHA256 14f5e992f8bb821ec1a9b197a7d3cce5e8b299c2c96036864e5b8b06434aeea3
MD5 304f2f1a6a993c552daba1edf70acfb4
BLAKE2b-256 ae52eb605446df6bb32b493e4166d872c894ab3f43f7664b099e9b44fc0fce90

See more details on using hashes here.

File details

Details for the file spotify_confidence-4.0.0-py3-none-any.whl.

File metadata

File hashes

Hashes for spotify_confidence-4.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 4fbd9020a13eb2697ed8b01586ffdf9e8fe7410b7a5147724cd1df20dc935351
MD5 64ca6608d666208c54d58162f5d7b0a5
BLAKE2b-256 3f80d3d35c43dd6319401f97f704d33d9cb085973af442c7e84486cbbba7d3b0

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page