fast-abtest 0.4.2

A fast and lightweight A/B testing library for Python.

Fast ABTest

A lightweight Python decorator for implementing A/B testing with automatic traffic distribution and built-in metrics monitoring. Compatible with FastAPI and works with both synchronous and asynchronous functions.

Installation

pip install fast-abtest

Quick Start

Basic Usage

from fast_abtest import ab_test, Metric

@ab_test(metrics=[Metric.LATENCY, Metric.ERRORS_TOTAL])
def recommendation_service(user_id: int) -> list[str]:
    # Main variant (A) - receives remaining traffic percentage
    return ["item1", "item2"]

@recommendation_service.register_variant(traffic_percent=30, disable_threshold=0.2)
def recommendation_service_b(user_id: int) -> list[str]:
    # Variant B - gets 30% of the traffic. If the error rate exceeds 0.2, traffic redirection will stop.
    return ["item3", "item4"]

FastAPI Integration

from fastapi import FastAPI, Depends
from fast_abtest import ab_test

app = FastAPI()

@app.get("/recommendations")
@ab_test(metrics=[])
async def get_recommendations(user_id: int):
    return {"items": ["A1", "A2"]}

@get_recommendations.register_variant(traffic_percent=30)
async def get_recommendations_b(user_id: int):
    return {"items": ["B1", "B2"]}

Important: For FastAPI, the route decorator (@app.get) must come before @ab_test.

Consistent distribution

from fastapi import FastAPI, Depends
from fast_abtest import ab_test

app = FastAPI()

@app.get("/recommendations")
@ab_test(metrics=[], consistency_key='user_id')
async def get_recommendations(user_id: int):
    return {"items": ["A1", "A2"]}

@get_recommendations.register_variant(traffic_percent=30)
async def get_recommendations_b(user_id: int):
    return {"items": ["B1", "B2"]}