fair-seldonian 3.0.0

Fairness-constrained machine learning using Seldonian algorithms with confidence bound optimizations

Fair-Seldonian

Fairness-constrained machine learning with high-confidence guarantees

---

A Python framework implementing the Quasi-Seldonian Algorithm (QSA) for training ML models that provably satisfy fairness constraints. Given a behavioral constraint and a confidence level δ, the algorithm either returns a model satisfying the constraint with probability ≥ 1 − δ, or returns No Solution Found — never an unsafe model.

Built on the Seldonian algorithm framework by Thomas et al. (2019), with extensions for tighter confidence bounds through constant-aware delta allocation, union bound optimization, and decomposed candidate-safety intervals.

Quick links

Documentation	parulgupta1004.github.io/fair-seldonian
Repository	github.com/parulgupta1004/fair-seldonian
Example notebook	examples/quickstart.ipynb
Paper	Thomas et al., Science 366 (2019) — doi:10.1126/science.aag3311

Installation

git clone https://github.com/parulgupta1004/fair-seldonian.git
cd fair-seldonian
uv sync                          # core dependencies
uv sync --extra experiments      # + Ray for parallel experiments
uv sync --extra plots            # + matplotlib for visualization
uv sync --extra notebook         # + JupyterLab to run examples/quickstart.ipynb

Or with pip:

pip install fair-seldonian
pip install "fair-seldonian[notebook]"          # JupyterLab + matplotlib to run the quickstart
pip install "fair-seldonian[experiments,plots]"

Usage

from fair_seldonian.algorithms import QSA
from fair_seldonian.models import eval_ghat
from fair_seldonian.data import get_data, data_split

data = get_data(N=10000, features=5, t_ratio=0.4,
                tp0_ratio=0.4, tp1_ratio=0.6, random_seed=42)
X_te, Y_te, T_te, X_tr, Y_tr, T_tr = data_split(
    frac=0.5, all_data=data, random_state=1, m_test=0.2)

theta, theta1, passed = QSA(X_tr, Y_tr, T_tr, "opt", None, None)

if passed:
    print("Upper bound:", eval_ghat(theta, theta1, X_te, Y_te, T_te, "opt"))
else:
    print("No Solution Found")

Custom configuration:

from fair_seldonian.config import SeldonianConfig
from fair_seldonian.constraints.inequalities import Inequality

config = SeldonianConfig(delta=0.01, inequality=Inequality.T_TEST, candidate_ratio=0.5)
theta, theta1, passed = QSA(X_tr, Y_tr, T_tr, "opt", None, None, config)

Examples

A runnable, end-to-end walkthrough lives in examples/quickstart.ipynb:

• generating synthetic data with a controllable fairness gap
• training with QSA and reading the high-confidence safety guarantee
• contrasting fair data (model certified) with unfair data (No Solution Found)
• decoding and customizing the postfix constraint, delta, and inequality
• comparing the five algorithm variants side by side
• visualizing accuracy vs. the certified fairness bound, with and without QSA, on the same dataset

Install the notebook dependencies and launch it with:

pip install "fair-seldonian[notebook]"
jupyter lab examples/quickstart.ipynb

View it rendered on nbviewer.

Algorithm variants

Mode	Description
base	Standard Hoeffding bound, uniform δ-splitting
mod	Decomposed candidate/safety estimation error
const	Constant-aware δ allocation
bound	Union bound optimization for repeated variables
opt	All optimizations combined

uv run python -m fair_seldonian.experiments.runner opt
uv run python -m fair_seldonian.experiments.plots

Citation

@software{fair_seldonian,
  author = {Parul Gupta},
  title  = {Fair Seldonian Framework},
  year   = {2020}
}

This work builds on:

Thomas, P.S., da Silva, B.C., Barto, A.G., Giguere, S., Brun, Y., & Brunskill, E. (2019). "Preventing undesirable behavior of intelligent machines." Science, 366(6468), 999–1004.

Contributing

Contributions are welcome! Please see CONTRIBUTING.md for guidelines, or open an issue to get started.

Contributors

Thanks to everyone who has contributed to this project!

License

MIT

---

Author: Parul Gupta