A model is one method that TabArena tunes under its shared protocol: shared preprocessing, a validation split provided by TabArena, a search space of up to 200 configurations, bagging under the arena's official validation protocol (TabArena: 8 folds x 1 set, asserted by the context), and the default / tuned / tuned + ensembled variants on the leaderboard. A system owns its whole pipeline (preprocessing, validation, tuning, ensembling) inside the budget TabArena hands it: AutoML frameworks such as AutoGluon, TabFM+, LLM agents, hosted APIs. If you would have to invent a search space for your method, it is a model. If that makes no sense because the method searches for itself, it is a system.

We accept methods their authors have already evaluated with the official pipeline and confirm the results by re-running them.

1. Say how your method differs from the entrants already on the leaderboard. A new version of an existing method goes into the existing folder and supersedes the old entry rather than becoming a new one.
2. Integrate it following the guide above and run the quick start.
3. Evaluate it yourself on TabArena-Lite (subset="lite", the first split of every dataset) with HPO where applicable (the default plus about 25 random configurations), or on the BeyondArena core subset. The quick starts run the official validation protocol by default; every result records the protocol it ran under, and a run made with official_validation_protocol=False is declared in the pull request.
4. Open a pull request; the template asks for the expected files, the results, the hardware and the entry-point script. You can also share the run's output directory (the expname folder with the results.pkl files) so we can verify and integrate the results directly.
5. A maintainer reviews the pull request, then runs the method on the full task set on the benchmark hardware for the final entry. We are happy to help with the integration and the run.
6. Maintainers verify that run against your Lite results, and the person you name in the template signs off on it, which marks the entry as verified.
7. The results are processed, hosted and registered, and the pull request is merged.
8. The leaderboard is regenerated from the hosted results after the merge, usually within days.

Questions go through the issue forms: one for model and system submissions, one for leaderboard or dataset questions that touch this code base. Pure leaderboard questions belong in the leaderboard's Community tab, dataset questions in Data Foundry. Anything else: mail@tabarena.ai.

Loads cached results and computes/plots leaderboards & metrics (ELO, win-rates, ranks). Depends on autogluon.tabular (not the full AutoGluon meta-package) — no model-fitting libraries and no torch.

uv pip install --prerelease=allow -e "./packages/tabarena[plot]"

Installs the core models used for standard benchmarking: tabpfn, tabicl, ebm, search_spaces, realmlp, tabdpt, tabm.

uv pip install --prerelease=allow -e "./packages/tabarena[benchmark]"

The extended extra is experimental and may fail to resolve or install due to incompatible version requirements across model dependencies. Use it only if you specifically need every model in a single environment; otherwise prefer benchmark or benchmark plus one specific model.

Layers the extended model set (modernnca, xrfm, sap-rpt-oss, ...) on top of the core benchmark set.

uv pip install --prerelease=allow -e "./packages/tabarena[benchmark,extended]"

To install only one extended model on top of benchmark (recommended over extended when you only need a single extra model), pass its extra by name — for example, just xrfm:

uv pip install --prerelease=allow -e "./packages/tabarena[benchmark,xrfm]"

Create a virtual environment in your workspace directory (it spans both repos cloned below, so .venv lives at the workspace root rather than inside either repo):

uv venv --seed --python 3.12 .venv
source .venv/bin/activate

Install editable AutoGluon and TabArena:

git clone https://github.com/autogluon/autogluon.git
./autogluon/full_install.sh

git clone https://github.com/autogluon/tabarena.git
uv pip install --prerelease=allow -e "./tabarena/packages/tabarena[benchmark]"

In PyCharm, mark packages/tabarena/src/ and each autogluon/src/ subdirectory as Sources Root so imports resolve.

tabarena and bencheval are published to PyPI as pre-releases for projects that cannot depend on git URLs, so pass --pre (pip) or --prerelease=allow (uv). The core package and [plot] are complete. Model extras whose upstream package is only available from git (tabfm, sap-rpt-oss, exaone_tabular) are empty on PyPI; the model's install hint tells you what to install by hand. The git checkout above stays the recommended install.

uv pip install --prerelease=allow "tabarena[plot]"   # or: pip install --pre "tabarena[plot]"
uv pip install --prerelease=allow bencheval          # leaderboard engine only

Add one of the following to your project's dependencies:

# TabArena depends on a pre-release of AutoGluon, so allow pre-releases when installing
# (e.g. `uv pip install --prerelease=allow ...` or `pip install --pre ...`).
# Alternatively, pin AutoGluon to a specific pre-release (an exact `==` pin resolves a
# pre-release without the flag), e.g. add `"autogluon.tabular==1.5.1b20260626"`.

# From PyPI (experimental pre-releases; each tabarena release pins its matching bencheval):
"tabarena>=0.1.0a1"
# From git (tip of main; publishable to PyPI only as a source-only extra, see issue #495):
"tabarena @ git+https://github.com/autogluon/tabarena.git#subdirectory=packages/tabarena"

Artifacts download to ~/.cache/tabarena/ by default. Override the location with the TABARENA_CACHE environment variable.

Raw data is ~100 GB per method type. Point TABARENA_CACHE at a large disk before downloading it.

The entry uses year=2026 because NeurIPS'25 proceedings are published in 2026.

@article{erickson2026tabarena,
  title   = {TabArena: A Living Benchmark for Machine Learning on Tabular Data},
  author  = {Erickson, Nick and Purucker, Lennart and Tschalzev, Andrej and Holzm{\"u}ller, David and Desai, Prateek and Salinas, David and Hutter, Frank},
  journal = {Advances in Neural Information Processing Systems},
  volume  = {38},
  year    = {2026}
}

@misc{purucker2026beyondiid,
  title         = {Beyond IID: How General Are Tabular Foundation Models, Really?},
  author        = {Purucker, Lennart and Tschalzev, Andrej and Erickson, Nick and Blayer, Gioia and Holzm{\"u}ller, David and Arazi, Alan and Pfefferle, Alexander and Tajjar, Mustafa and Varoquaux, Ga{\"e}l and Hutter, Frank},
  year          = {2026},
  eprint        = {2606.30410},
  archivePrefix = {arXiv},
  primaryClass  = {cs.LG},
  url           = {https://arxiv.org/abs/2606.30410}
}