Fairical

Fairical is a Python library for rigorously evaluating and comparing demographically fair machine-learning systems through the lens of multi-objective optimization. Rather than treating fairness as a single constraint, Fairical recognizes that real-world deployments must balance multiple, often conflicting fairness metrics (e.g., demographic parity, equalized odds across race, gender, age) alongside traditional utility measures like accuracy. It implements a model-agnostic evaluation framework that approximates Pareto fronts of utility-fairness trade-offs, then distills each system's performance into a compact measurement table and radar chart. By calculating convergence (how close models get to optimal trade-offs), diversity (uniform distribution and spread of solutions), capacity (number of non-dominated points), and a unified convergence-diversity score via hypervolume, Fairical delivers both quantitative rigor and qualitative clarity.

For installation and usage instructions, check-out our documentation.

If you use this library in published material, we kindly ask you to cite this work:

@article{ozbulak_multi-objective_2025,
	title={A Multi-Objective Evaluation Framework for Analyzing Utility-Fairness Trade-Offs in Machine Learning Systems},
	author={Özbulak, Gökhan and Jimenez-del-Toro, Oscar and Fatoretto, Maíra and Berton, Lilian and Anjos, André},
	journal={Machine Learning for Biomedical Imaging},
	volume={3},
	number={Special issue on FAIMI},
	pages={938--957},
	doi={10.59275/j.melba.2025-ab9a},
	year={2025}
}