RouteFM

Official Python package for Pretrain Once, Route Anywhere: Towards a Foundation Model for LLM Routing.

RouteFM is a pretrained router that adapts a frozen model to new tasks and candidate pools through behavioral Context. Candidate identities are not used as features: each candidate is characterized by a small number of observed query, quality, and relative-cost tuples.

Install

RouteFM requires Python 3.10 or newer. Install the BGE text interface with:

python -m pip install "routefm-router[bge]"

The RouteFM package is lightweight; released routing checkpoints remain on Hugging Face and are downloaded and checksum-verified on first use. The built-in text API also pins the compatible BGE encoder revision.

Route text in Python

from routefm import RouteFMRouter

router = RouteFMRouter.from_pretrained(encoder="bge", device="cpu")
router.set_context({
    "small-model": [
        {"query": "What is 2 + 2?", "score": 1.0, "cost": 0.01},
        {"query": "Summarize this paragraph.", "score": 0.7, "cost": 0.02},
    ],
    "large-model": [
        {"query": "What is 2 + 2?", "score": 1.0, "cost": 0.10},
        {"query": "Summarize this paragraph.", "score": 0.95, "cost": 0.20},
    ],
})

decision = router.route("Prove that there are infinitely many primes.")
print(decision.model_name)
print(decision.predicted_scores)

The default decision rule selects the highest predicted quality. Predicted cost is relative rather than calibrated currency or latency and is not used by that rule. No RouteFM parameter update occurs.

For precomputed BGE or Qwen multimodal embeddings, use RouteFMRouter.predict_arrays(...). The existing routefm-predict CLI and .npz workflow remain supported.

Resources

• Paper
• Source and documentation
• Frozen checkpoints and model card
• Custom-data schema
• Pretraining guide

Citation

@misc{lai2026pretrainoncerouteanywhere,
  title         = {Pretrain Once, Route Anywhere: Towards a Foundation Model for LLM Routing},
  author        = {Guannan Lai and Han-Jia Ye},
  year          = {2026},
  eprint        = {2609.37362},
  archivePrefix = {arXiv},
  primaryClass  = {cs.AI},
  url           = {https://arxiv.org/abs/2609.37362}
}

RouteFM source code and released routing weights use the Apache License 2.0. Third-party encoders and datasets retain their own licenses and terms.