CallBus

Tool retrieval for LLM agents with hundreds of tools. Instead of putting every tool definition into the prompt, callbus picks the few a request needs, with an embedding model (Qwen3-Embedding-8B, trained further on tool retrieval), and serves them to your agent over MCP or REST. Every retriever it ships is measured on the same public benchmarks.

Status: alpha (0.2); interfaces may still change. Documentation: https://yaman.dev/callbus/

Quick start

Index MCP servers or OpenAPI specs, then serve them to any MCP client as two tools, search_tools and call_tool:

pip install "callbus[mcp]"
CallBus ingest mcp --server time="uvx mcp-server-time" --out tools/
CallBus serve --data tools/                          # MCP at http://127.0.0.1:8765/mcp, REST at /v1

CallBus ranks with its backbone, Qwen3-Embedding-8B trained further on tool retrieval (yasinyaman/callbus-emb-8b), served by vLLM (--emb-url, by default http://127.0.0.1:8091/v1). On a GPU host, Docker runs both:

cd deploy/docker && cp .env.example .env              # set CALLBUS_API_KEY
docker compose run --rm callbus ingest mcp --config /config/callbus.json --out /data
docker compose up -d

The quick start connects Claude Code, Claude Desktop and REST clients. No big GPU? The backbone also runs as a GGUF in Ollama, at the same measured quality. Guide

Results

Retrieval quality on three benchmarks, every number from callbus eval under ToolRet's protocol (the reports are in docs/results/; scripts/readme_table.py checks each one before it prints it).

• ToolRet: 7,961 queries over 44,453 tools, top 100 over the whole corpus. NDCG@10 is the micro-average of the paper's released code; cat-macro is the paper's own aggregation (the mean of the web, code and customized categories) and the only column with published numbers. Our BM25 reproduces the paper's BM25s within 0.1 (22.24 / 36.41 against 22.32 / 36.46).
• LiveMCPBench (94 queries, 525 tools) and MCP-Zero (2,792 tools): the tool text includes the MCP server's name (callbus data server-names). One LiveMCPBench query is about one point. MCP-Zero ships no queries: ours were written by Qwen3-8B, one per tool (callbus data pull mcp-zero), so its column does not compare with the MCP-Zero paper. Top-1 is Precision@1. LiveMCPBench shows Recall@5; on NDCG@10 the v0.2 backbone leads the v0.1 heads (55.74 / 53.95), on Recall@5 it trails them (52.06 / 53.03) — the gap either way is about one query.
• With instruction, each query carries its task's instruction (ToolRet) or a generic one (the MCP sets), as the embedding model is served; the generic instruction costs BM25 on the MCP sets. BM25 is bm25s without stemming, the paper's setting.
• The heads (29.9M parameters, docs/heads/MODEL_CARD.md) were trained on ToolRet's training pairs, so ToolRet is in-domain for them and the MCP sets are not. On MCP-Zero, BM25 without instruction still wins at top-1: each generated query opens with a server: line that usually names the server, and exact matching rewards that.
• The callbus backbone v0.2 (docs/backbone/MODEL_CARD.md) is Qwen3-Embedding-8B with a LoRA trained on 20,000 of ToolRet's training pairs, served without heads (the v0.1 heads cost it 0.2–3.9 points: ToolRet micro least, LiveMCPBench Recall@5 most). Its checkpoint was picked on MCP-Zero, so that column is its selection set; ToolRet is in-domain, LiveMCPBench is held out. Beyond these sets, on generated requests over a GitHub + Stripe catalogue it gains 12–20 NDCG@10 points on tasks that need two or three tools and ties with the base model on requests for one tool (the model card has the numbers).
• FP8: the bf16 weights quantized as vLLM loads them (--quantization fp8). Every column is within a point of bf16, at half the weight memory (7.92 GiB against 14.11) and a third off batch-1 latency (56.1 ms against 92.4, cold, over the 44,453 ToolRet tools). The numbers hold on the image the Docker examples ship, vllm/vllm-openai:v0.30.0, as well as on the NGC image they were measured on: the two stacks' vectors agree to a cosine of 0.997 and rank within half a point of each other on all three sets.
• Reproduce: bash scripts/readme_results.sh in a clone, where the backbone is served (reports in docs/results/; EMB_URL, EMB_MODEL, TAG and ROWS select another endpoint and rows, HEADS the v0.1 heads), then uv run python scripts/readme_table.py --write; benchmarks has the commands.

What's inside

• Ingestion of MCP servers (stdio and streamable HTTP) and OpenAPI 3.x specs; a re-run syncs only what changed. Guide
• Search and serve: adaptive K, a persistent vector index (numpy, FAISS HNSW or pgvector), an MCP proxy with two tools, a REST API, API keys, Prometheus metrics and a usage log; an optional second scorer reranks the shortlist (serve --rerank). Guide
• Agent platforms: callbus as Claude's (tool_reference) and OpenAI's (client-side tool_search) tool search. Guide Agents with only a shell get a skill. Guide
• Frameworks: LangChain agents (create_agent middleware), LangGraph (langgraph-bigtool), LlamaIndex agents, the LiteLLM proxy and Strands Agents (over MCP, no adapter). Guide
• Fine-tuning: heads trained on your own request-to-tool pairs, the epoch picked on a dev set. Guide
• Learning from usage: callbus learn mines the serve log into training pairs and trains heads from them; callbus ab judges the candidate on live traffic and promotes or rolls back, per tenant. Guide
• Benchmarks: ToolRet, LiveMCPBench and MCP-Zero with BM25, dense and head scorers (and CLM, for comparison). Benchmarks CLM here is Contrastive-LM, whose head architecture callbus's heads follow, not Context Language Models.
• Docker and Helm: the callbus image for amd64 and arm64, compose files with vLLM, a Dockerfile that puts vLLM and callbus in one container, and a Helm chart. Guide

Why

• Tool definitions are expensive context. Anthropic measured 58 tools at about 55K tokens per request and reports tool-selection accuracy falling past 30 to 50 tools. RAG-MCP lifted selection accuracy from 13.6% to 43.1% on a large MCP set by retrieving tools first.
• Hosted tool searches are tied to one model provider or cloud, and most are lexical. callbus is model-agnostic and runs on your own hardware.
• An embedding backbone trained further on tool retrieval (yasinyaman/callbus-emb-8b, the default). You embed your tools once and rank with one dot product. Optional small heads on top of it (29.9M parameters for the request and the tool side together) start as the identity, so heads fine-tuned on your data start from the base model's quality, not below it.

Feedback

Tried it? Tell us how it went: what you set up, what worked and what did not. Questions and ideas go to Discussions.

Contributing

Set-up, tests and conventions are in CONTRIBUTING.md. The weekly reports behind every number, in Turkish, are in docs/reports/.

License

Apache-2.0 (LICENSE). NOTICE credits what callbus takes from CLM (the head architecture), ToolRet (its task metadata) and MCP-Zero (its query prompts); THIRD_PARTY_NOTICES.md lists the dependencies, models and container images. Contributions are welcome: see CONTRIBUTING.md and the code of conduct; report vulnerabilities as SECURITY.md says.