Symbolic Regression Benchmark Framework

PyPI · Docs · Interactive results

srbf: Symbolic Regression Benchmark Framework

Publications

• Saegert & Köthe 2026, Breaking the Simplification Bottleneck in Amortized Neural Symbolic Regression (ICML 2026) https://arxiv.org/abs/2602.08885

srbf evaluates symbolic-regression models on shared benchmarks with shared metrics. It is the Symbolic Regression Benchmark Framework carved out of flash-ansr: the Benchmark driver, model adapters, and metrics, over symbolic-data catalogs. It depends one-way on flash-ansr (srbf imports flash-ansr; flash-ansr never imports srbf).

Built for contributions. Developers of SR methods add their model with an adapter that runs in the method's own environment (any torch, simplipy or Julia version): srbf new mymethod writes the worker, its suite config, an environment recipe and a test; srbf check runs it on real problems; srbf run and srbf analyze give the numbers. The pull request carries those files. Methods compatible with srbf's pins can instead register an in-process adapter class. The built-in adapters (flash_ansr, pysr, nesymres, e2e, lample_charton, brute_force) are reference examples, not a closed set. See the adapter contribution guide and CONTRIBUTING.md.

The benchmark seam (srbf.core Protocols + the Benchmark driver) is model-agnostic, the data source is always a symbolic-data catalog, and adapters are a thin mapper over each model (flash-ansr via FlashANSR.infer()). Inline !sweep config cross-products and multi-draw bootstrap reporting (bootstrap_report / draw_distribution) are built in.

Install

pip install srbf                 # benchmark driver + metrics + the flash-ansr adapter (usable out of the box)
pip install "srbf[baselines]"    # + PySR and other pip baseline deps (sympy, pysr, omegaconf)

srbf pulls in flash-ansr, symbolic-data, and simplipy automatically, and requires Python >= 3.12. The PySR adapter ships in the base wheel but the pysr package (plus a Julia precompile) comes with the [baselines] extra, so a bare install does not include a runnable PySR baseline. The unpackaged research baselines (NeSymReS, E2E) are provisioned out-of-band; see docs/models.md.

Quickstart

# 1. point srbf at a tree holding configs/, data/, and models/ (your srbf checkout works)
export FLASH_ANSR_ROOT=$(pwd)

# 2. get a model to evaluate (flash-ansr's CLI ships with srbf)
flash_ansr install psaegert/flash-ansr-v25.0-T7-3M

# 3. run an evaluation. The config names a symbolic-data catalog (`fastsrb`); it is fetched from
#    Hugging Face on first use and cached, so there is no local data-build step. The config is a
#    sweep over candidate counts; --sweep-filter picks one rung for a smoke test.
srbf run -c configs/evaluation/scaling/flash-ansr-v25.0-T7-3M_fastsrb.yaml --sweep-filter ladder=32 --limit 50 -v

Outputs land under results/evaluation/.../*.pkl, one row per evaluated problem with the raw prediction columns (derive FVU / recovery / F1 in a separate step; see docs/running.md). Run programmatically instead:

from srbf import Benchmark

# A config with inline !sweep / experiments expands to several runs; expand and run each one.
for benchmark in Benchmark.runs_from_config("configs/evaluation/scaling/flash-ansr-v25.0-T7-3M_fastsrb.yaml"):
    benchmark.run()  # resume-aware; a no-op if that run's configured target is already reached

# For a single, fully-resolved run (no !sweep / experiments), use from_config directly:
# Benchmark.from_config(config_dict).run()

Documentation

Development

pip install -e ".[dev]"
pre-commit run --all-files
pytest tests

License

MIT (see LICENSE). Third-party attributions in THIRD_PARTY_LICENSES.