Skip to main content

tiebreak

Ollama serves chat and embeddings but has no rerank endpoint (ollama#3368, open, 290 thumbs-up on 2026-11-10). tiebreak is the missing piece for a local RAG stack: a CPU-only cross-encoder server, ONNX Runtime with no torch, that answers the Cohere and Jina /v1/rerank request shapes so existing clients point at it with a base URL change.

The opinion: no reranker is "better" in general, and a shim that ships one without showing where it loses is selling a guess. tiebreak ships the fast model as the default, publishes the dataset where that default makes ranking worse, and ships the bench script so the choice is made on your data. Every response also says which documents were cut to fit the model.

Measured

Three BEIR test sets, BM25 top-100 candidates, the identical cached pool for every model, nDCG@10. One laptop CPU (Windows, onnxruntime CPU provider). Raw rows: bench/results/ndcg.jsonl; rerun with python bench/ndcg.py --model <alias> --queries 100 --dataset scifact|nfcorpus|fiqa.

Model (alias) Dataset Queries BM25 Reranked Change ms per pair
minilm (default, ms-marco-MiniLM-L-6-v2 fp32) SciFact 100 0.7303 0.7218 -0.009 55.4
minilm-int8 SciFact 100 0.7303 0.7178 -0.013 40.6
bge-reranker-base-int8 SciFact 100 0.7303 0.7501 +0.020 174.8
minilm (default) NFCorpus 100 0.3395 0.3871 +0.048 45.2
bge-reranker-base-int8 NFCorpus 100 0.3395 0.3431 +0.004 171.9
bge-reranker-base fp32 (Xenova onnx/model.onnx) NFCorpus 100 0.3395 0.3430 +0.004 210.0
minilm (default) FiQA 100 0.2074 0.3164 +0.109 33.4
bge-reranker-base-int8 FiQA 100 0.2074 0.3053 +0.098 110.0
minilm (default) SciFact 30 0.7409 0.7211 -0.020 46.6
bge-reranker-base-int8 SciFact 30 0.7409 0.7776 +0.037 177.3
bge-reranker-v2-m3 (int8) SciFact 30 0.7409 0.8264 +0.085 492.2

What the table says:

  • MiniLM lowered nDCG@10 on SciFact (scientific claims, where BM25 is already strong; 13% of its pairs were cut at 512 tokens) and raised it most of any model run on NFCorpus (medical questions) and FiQA (finance questions). It is the default because it is 3 to 4 times faster than bge-base and its average gain over the three sets is larger (+0.049 against +0.041).
  • bge-reranker-base never went below BM25 here, but it gained almost nothing on NFCorpus, and that is the model, not quantization: the fp32 file scores 0.3430 against int8's 0.3431.
  • bge-reranker-v2-m3 had the largest gain measured, on 30 queries of one dataset only, at about 10 times MiniLM's cost.
  • 30-query rows move a lot: bge-base went from +0.037 on 30 SciFact queries to +0.020 on 100.

Speed in practice: reranking 20 candidates costs about 1 s with minilm, about 3.5 s with bge-reranker-base-int8, about 10 s with bge-reranker-v2-m3 on this machine. Before trusting any of them, run bench/ndcg.py or your own labelled queries and keep the model only if it beats your first-stage retriever.

bench/runtime.py checks the ONNX path against the torch reference for MiniLM: fp32 logits match exactly (max diff 0.0); the int8 file drifts up to 0.74 logits and keeps only 1 in 10 top-10 positions identical, which is why int8 MiniLM is opt-in.

Run

pip install tiebreak-rerank        # the PyPI name; the command and import are `tiebreak`
tiebreak serve                      # 127.0.0.1:11435, downloads the default model once
tiebreak serve --model bge-reranker-v2-m3
tiebreak rerank "what lowers blood pressure" "The Eiffel Tower is in Paris." "ACE inhibitors relax blood vessels."

A live call against tiebreak serve --model bge-reranker-base-int8:

$ curl -s 127.0.0.1:11435/v1/rerank -H 'content-type: application/json' -d '{"query":"what lowers blood pressure","documents":["The Eiffel Tower is in Paris.","ACE inhibitors reduce blood pressure by relaxing blood vessels.","Salt intake raises blood pressure in sensitive people."],"top_n":2,"return_documents":true}'
{"id":"...","model":"Xenova/bge-reranker-base","results":[{"index":1,"relevance_score":0.983...,"document":{"text":"ACE inhibitors reduce blood pressure by relaxing blood vessels."}},{"index":2,"relevance_score":0.582...,"document":{"text":"Salt intake raises blood pressure in sensitive people."}}],"usage":{"total_tokens":59,"prompt_tokens":59},"meta":{"api_version":{"version":"2"},"score_scale":"sigmoid","truncated_indices":[]},"warnings":[]}

The same handler is mounted on /v1/rerank, /v2/rerank, /rerank and /api/rerank. Documents may be strings or {"text": ...} objects. GET /health and GET /v1/models exist for probes and clients that list models. The Cohere Python SDK works against it unchanged (cohere.ClientV2(base_url="http://127.0.0.1:11435", api_key="unused"); covered by a model test).

Model aliases: minilm, minilm-int8, bge-reranker-base, bge-reranker-base-int8, bge-reranker-v2-m3, jina-reranker-v1-tiny-en. Anything else is read as a Hugging Face repo id or a local directory holding tokenizer.json and an ONNX file (--onnx-file picks which). Only the chosen weights file and the tokenizer are downloaded, never the whole repo.

Limits and safety

Every cap is an environment variable and every refusal names the field and the limit:

Variable Default Refusal
TIEBREAK_MAX_DOCUMENTS 1000 413, documents
TIEBREAK_MAX_CHARS_PER_DOCUMENT 50000 413
TIEBREAK_MAX_TOTAL_TOKENS 400000 413
TIEBREAK_TIMEOUT_S 30 504
TIEBREAK_MAX_LENGTH 512 truncates, listed in meta.truncated_indices and warnings

The server binds 127.0.0.1. It refuses to start on any other address unless TIEBREAK_API_KEY is set, and then every request needs Authorization: Bearer <key>. Scores are sigmoid of the logit by default (0 to 1, like Cohere); TIEBREAK_SCORES=raw returns logits.

Tests

pytest -q            # 18 passed: request shapes, caps, auth, refusal text (no model download)
pytest -q -m model   # 4 passed: real scoring, truncation reporting, Cohere SDK round trip, CLI

Not done

No GPU provider, no streaming, no batching across concurrent requests. Quality numbers cover three small BEIR sets at 100 queries (bge-reranker-v2-m3 at 30), not the full test sets.

MIT licence.

Metadata

Release files for tiebreak-rerank 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for tiebreak-rerank 0.1.0
File Size Uploaded Provenance Actions
tiebreak_rerank-0.1.0.tar.gz 123.5 kB Without provenance Details for tiebreak_rerank-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for tiebreak-rerank 0.1.0
File Interpreter ABI Platform Actions
tiebreak_rerank-0.1.0-py3-none-any.whl Python 3 none any Details for tiebreak_rerank-0.1.0-py3-none-any.whl

Total release size: 136.4 kB

Release files / tiebreak_rerank-0.1.0.tar.gz

Download URL tiebreak_rerank-0.1.0.tar.gz
Size 123.5 kB
Tags Source
SHA-256 checksum
How to use checksums
db43b16f947462e0549710388e97ec4b0403a091a479776a2960dc188c8e6288
BLAKE2b-256 checksum
How to use checksums
cfc2051b960bc176ed2921821f0418c1bbc681b17de799395a0d788850852a2e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.11.24 {"installer":{"name":"uv","version":"0.11.24","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":null,"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

Release files / tiebreak_rerank-0.1.0-py3-none-any.whl

Download URL tiebreak_rerank-0.1.0-py3-none-any.whl
Size 13.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
d198eab3c480ac3fe35319b644d69b27b06f02ac3c02337c30c9a23edd540424
BLAKE2b-256 checksum
How to use checksums
ad06c543dc5308604c9b157602134c97b96776c349076485a98359c2cfb8c7fe
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.11.24 {"installer":{"name":"uv","version":"0.11.24","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":null,"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

Security for tiebreak-rerank 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page