mixedbench

A benchmark of posterior approximation and prediction for generalized linear mixed models (GLMMs) on real grouped data.

What and why

• Bayesian methods for GLMMs are hard to compare: each paper and released method tests them on its own few datasets. This benchmark aims to fix that.
• mixedbench lists the documented mixed-effects models of 641 public tabular datasets with grouping structure, each downloaded at a fixed version.
• A fixed set of rules turns each dataset into an inference problem ready to fit. Results of different methods are comparable.
• From release 1.1, suites add reference posteriors to score against.

Install

pip install mixedbench

No R needed: every dataset loads in Python.

Quickstart

from pathlib import Path

from mixedbench import registry
from mixedbench.contract import Problem, load, materialise

cache = Path('cache')
[row] = [row for row in registry.rows() if row.id == 'lme4/sleepstudy/1']
table = registry.Tables(cache).load(row)  # fetched from CRAN at its pin
materialise(Problem.build(row, table), cache)
problem = load('lme4/sleepstudy/1', cache)  # later, offline

Check out the demo for a friendly introduction.

Contents

• The package mixedbench: registry (the registry files, summarised in SUMMARY.md), datasets (the loader) and contract (Problem, materialise, load).
• suite/1.1/candidates.csv: the candidate problems of suite 1.1 with their contract hashes; refused.csv: the rows left out, each with its reason.
• survey/: the pipeline that built the registry and its record; provenance, not needed to use the package.

Contributing

Interested in becoming part of mixedbench? Check out CONTRIBUTING.md.

Citing

Please cite the release you used: CITATION.cff, or click "Cite this repository" on GitHub. All versions share the DOI 10.5281/zenodo.23285683.

Licence

• Code: MIT. Registry and suite files: CC BY 4.0.
• The data keep their sources' terms: they are fetched at first use, not redistributed.