Relational Transformer (RT)

This repo is the official implementation of the Relational Transformer (RT) architecture and the recipes to pretrain, fine-tune, inference and test-time scale Relational Foundation Models (RFMs) based on RT.

The following results are fully reproducible from here:
🥇 RT-J is the #1 in-context model on the RelBench leaderboard at the time of submission (in-context, fine-tuned).
🥇 RT-PluRel is #1 on the combined system + model leaderboard on RelArena-alpha at the time of submission (in-context, fine-tuned).

This repo is linked to the following papers in the Stanford Tabular and Relational (STAR) project ecosystem:

Models and datasets (including preprocessed versions) are available on Hugging Face.

Installation

pip install relational-transformer

This installs the rt package, including the Rust sampler rt.rustler, without any dependency on Rust.

Quickstart

examples/quickstart.ipynb (open in Colab) evaluates the released RT-J checkpoint on a RelBench task, rel-f1/driver-dnf, with no training: download one preprocessed database, predict every test row from its sampled context, score the AUROC. A few minutes on a GPU runtime.

Bring your own database

The BYOD notebook (open in Colab) preprocesses your database and predicts tasks you define in SQL with a released checkpoint. For more than a first look, examples/ runs the full recipes on your own data in RelBench format: in-context prediction with a frozen checkpoint, or per-task fine-tuning.

Development

We use pixi to manage one self-contained environment (Python, PyTorch + CUDA, Rust, and all dependencies), built on first use.

git clone https://github.com/stanford-star/relational-transformer.git
cd relational-transformer
pixi run pytest                              # the test suite
pixi run python -m examples.eval.plan        # or examples.pretrain.phases, ...

Examples

examples/ holds the recipes behind the released checkpoints and leaderboard entries, with the notes on how each part works.

Citation

If you use this repo, please cite the following papers:

@inproceedings{ranjan2026rtj,
    title={{RT-J}: Large-Scale Pretraining of Relational Transformers for Context-Efficient Predictions},
    author={Rishabh Ranjan and Vignesh Kothapalli and Harshvardhan Agarwal and Charilaos Kanatsoulis and Roshan Upendra and Tom Palczewski and Carlos Guestrin and Jure Leskovec},
    booktitle={The Fortieth Annual Conference on Neural Information Processing Systems},
    year={2026}
}

@inproceedings{ranjan2026relational,
    title={Relational Transformer: Toward Zero-Shot Foundation Models for Relational Data},
    author={Rishabh Ranjan and Valter Hudovernik and Mark Znidar and Charilaos Kanatsoulis and Roshan Upendra and Mahmoud Mohammadi and Joe Meyer and Tom Palczewski and Carlos Guestrin and Jure Leskovec},
    booktitle={The Fourteenth International Conference on Learning Representations},
    year={2026}
}

Additionally, if you use RT-PluRel please also cite:

@inproceedings{kothapalli2026plurel,
    title={{PluRel}: Synthetic Data unlocks Scaling Laws for Relational Foundation Models},
    author={Vignesh Kothapalli and Rishabh Ranjan and Valter Hudovernik and Vijay Prakash Dwivedi and Johannes Hoffart and Carlos Guestrin and Jure Leskovec},
    booktitle={Forty-third International Conference on Machine Learning},
    year={2026}
}

License

MIT, see LICENSE. The released checkpoints and datasets carry their own licences; "the Join" is assembled from third-party databases that keep theirs.