BanditLab

A modular framework for experimenting with multi-armed bandits (MAB).

• 20+ algorithms (from classical to state-of-the-art)
• unified API
• plug-and-play models and datasets
• config-driven experiments

BanditLab is designed for both research and practical experimentation. It provides a unified interface for combining:

• bandit algorithms (UCB, Thompson Sampling, Neural, GP-based, etc.)
• predictive models (linear, GLM, neural networks, Gaussian processes)
• environments (real datasets or simulators)

---

Installation

pip install BanditLab

---

Quick Start (Python API)

from mab_framework.algorithms import ThompsonSampling
from mab_framework.environments import DatasetEnvironment

env = DatasetEnvironment("data/mushroom_bandit_5000.csv")

bandit = ThompsonSampling(...)

for context in env:
    arm = bandit.select_arm(context)
    reward = env.pull(arm)
    bandit.update(context, arm, reward)

---

Config-Based Experiments (Recommended)

BanditLab supports fully declarative experiment setup via configs.

Example config:

experiment:
  name: "pool_test"
  steps: 200
  n_runs: 5

environment:
  name: "DatasetEnvironment"
  params:
    dataset_path: "data/E1_dataset.npz"

algorithms:
  - name: "ThompsonSampling"
    display_name: "Thompson Sampling (TS)"
    params: {}
    model:
      name: "OnlineRidgeRegression"
      params: { l2_reg: 1.0 }
      one_model_per_arm: true

  - name: "UCBAlgorithm"
    display_name: "LinUCB (alpha=1.0)"
    params: { alpha: 1.0 }
    model:
      name: "OnlineRidgeRegression"
      params: { l2_reg: 1.0 }
      one_model_per_arm: true

metrics:
  - cumulative_regret
  - average_regret

output:
  save_path: "./results/pool_test"

Run via:

python banditlab config.yaml

This allows running experiments without writing Python code and ensures full reproducibility.

---

Key Features

• 20+ algorithms — from classical (UCB, TS) to neural and GP-based methods
• Model–Algorithm decoupling — combine any algorithm with any reward model
• Config-driven experiments — easy experimentation without coding
• Contextual bandits support
• Delayed feedback support — built-in support for bandits with delays
• Extensible — easily implement new algorithms or models
• Reproducible experiments — runner, logging, and metrics included

---

Core Design

BanditLab separates decision-making from prediction:

• Models learn to predict rewards from context
• Algorithms decide which arm to pull using model outputs

This enables flexible combinations:

• Thompson Sampling + Linear Model
• Thompson Sampling + GLM
• UCB + Neural Network
• UCB + Gaussian Process

---

Architecture Overview

The framework is built around four components:

• Environments — provide contexts and rewards
• Models — estimate reward (typically one per arm)
• Algorithms — handle exploration vs exploitation
• Runner — executes experiment loops

---

Example: Running a Benchmark

python scripts/run_mushrooms.py

This runs multiple algorithms on a real dataset and produces:

• cumulative regret plots
• average regret curves

---

Supported Methods

Algorithms

Includes 20+ implementations, such as:

• Epsilon-Greedy
• UCB / LinUCB
• Thompson Sampling
• Neural UCB
• GP-based methods
• GLM-based bandits

🕰️ Academic Delayed Feedback Algorithms

BanditLab provides state-of-the-art academic implementations of Multi-Armed Bandit algorithms under delayed feedback. These methods strictly adhere to the mathematical bounds established in peer-reviewed literature and avoid ad-hoc heuristics.

• JoulaniDelayedUCB: Implements the order-optimal delay-adapted UCB from Joulani et al., 2013 ("Online Learning under Delayed Feedback"). It strictly computes confidence intervals based exclusively on resolved observations, handling arbitrary delays elegantly.
• VernadeDelayedUCB: From Vernade et al., 2017 ("Stochastic Bandit Models for Delayed Conversions"). This algorithm assumes knowledge of the delay CDF and models the expected number of arrivals. Our implementation uses amortized $O(1)$ complexity via $D_{max}$ truncation, avoiding the traditional $O(T^2)$ computational bottleneck.
• PatientBandits: Derived from Manegueu et al., 2020 ("Stochastic bandits with arm-dependent delays"). It guarantees finite-horizon theoretical bounds specifically crafted for heavy-tailed delay distributions and relies on a dedicated tail-index parameter $\alpha$.
• DelayedThompsonSampling: As established by Chapelle & Li, 2011, plain Thompson Sampling handles delays natively without modifying the posterior. This algorithm simply waits for resolved data to update the model and samples from the true posterior, sidestepping variance deflation pitfalls.

Models

• Linear / Ridge Regression
• GLM (Laplace approximation)
• Gaussian Processes (RFF)
• Neural Networks
• LASSO-based models

---

Project Structure

mab_framework/
├── algorithms/
├── models/
├── environments/
├── experiment/
└── scripts/

---

Extending the Framework

Custom Model

predict(context)
update(context, reward)

Custom Algorithm

select_arm(context)
update(context, arm, reward)

All components inherit from base classes, making extension straightforward.

---

Reproducibility

BanditLab includes:

• experiment runner
• logging utilities
• regret metrics

Designed for fair comparison of algorithms across datasets.

---

Documentation

Detailed developer documentation is available in:

docs/DEVELOPMENT.md

---

License

MIT License

---

Citation

If you use BanditLab in research, please consider citing the repository.