Semantic Router is a superfast decision-making layer for your LLMs and agents. Rather than waiting for slow LLM generations to make tool-use decisions, we use the magic of semantic vector space to make those decisions — routing our requests using semantic meaning.
Read the Docs
Versions
Semantic Router has two release lines. The 0.x line (v0 branch) is the current stable library and finishes at 0.3.0, after which it receives bug fixes only. The 1.x line (main branch) is a breaking rewrite of the routing layer and ships as 1.0.0. Pin to the line you are on:
pip install "semantic-router<1" # 0.x
Details, and the docs for each line, are on the versions page.
Quickstart
To get started with semantic-router we install it like so:
pip install -qU semantic-router
❗️ If wanting to use a fully local version of semantic router you can use HuggingFaceEncoder and LlamaCppLLM (pip install -qU "semantic-router[local]", see here). To use the HybridRouteLayer you must pip install -qU "semantic-router[hybrid]".
We begin by defining a set of Route objects. These are the decision paths that the semantic router can decide to use, let's try two simple routes for now — one for talk on politics and another for chitchat:
from semantic_router import Route
# we could use this as a guide for our chatbot to avoid political conversations
politics = Route(
name="politics",
utterances=[
"isn't politics the best thing ever",
"why don't you tell me about your political opinions",
"don't you just love the president",
"they're going to destroy this country!",
"they will save the country!",
],
)
# this could be used as an indicator to our chatbot to switch to a more
# conversational prompt
chitchat = Route(
name="chitchat",
utterances=[
"how's the weather today?",
"how are things going?",
"lovely weather today",
"the weather is horrendous",
"let's go to the chippy",
],
)
# we place both of our decisions together into single list
routes = [politics, chitchat]
We have our routes ready, now we initialize an embedding / encoder model. We currently support a CohereEncoder and OpenAIEncoder — more encoders will be added soon. To initialize them we do:
import os
from semantic_router.encoders import CohereEncoder, OpenAIEncoder
# for Cohere
os.environ["COHERE_API_KEY"] = "<YOUR_API_KEY>"
encoder = CohereEncoder()
# or for OpenAI
os.environ["OPENAI_API_KEY"] = "<YOUR_API_KEY>"
encoder = OpenAIEncoder()
With our routes and encoder defined we now create a RouteLayer. The route layer handles our semantic decision making.
from semantic_router.routers import SemanticRouter
rl = SemanticRouter(encoder=encoder, routes=routes, auto_sync="local")
We can now use our route layer to make super fast decisions based on user queries. Let's try with two queries that should trigger our route decisions:
rl("don't you love politics?").name
[Out]: 'politics'
Correct decision, let's try another:
rl("how's the weather today?").name
[Out]: 'chitchat'
We get both decisions correct! Now lets try sending an unrelated query:
rl("I'm interested in learning about llama 2").name
[Out]:
In this case, no decision could be made as we had no matches — so our route layer returned None!
Integrations
The encoders of semantic router include easy-to-use integrations with Cohere, OpenAI, Hugging Face, FastEmbed, and more — we even support multi-modality!.
Our utterance vector space also integrates with Pinecone and Qdrant!
📚 Resources
Docs
| Notebook | Description |
|---|---|
| Introduction | Introduction to Semantic Router and static routes |
| Dynamic Routes | Dynamic routes for parameter generation and functionc calls |
| Save/Load Layers | How to save and load RouteLayer from file |
| LangChain Integration | How to integrate Semantic Router with LangChain Agents |
| Local Execution | Fully local Semantic Router with dynamic routes — local models such as Mistral 7B outperform GPT-3.5 in most tests |
| Route Optimization | How to train route layer thresholds to optimize performance |
| Multi-Modal Routes | Using multi-modal routes to identify Shrek vs. not-Shrek pictures |
| Healthcare Administrative Routing | Fully local routing, threshold optimization, and evaluation across seven synthetic healthcare administrative workflows |
Online Course
Community
- Dimitrios Manias, Ali Chouman, Abdallah Shami, Semantic Routing for Enhanced Performance of LLM-Assisted Intent-Based 5G Core Network Management and Orchestration, IEEE GlobeCom 2024
- Julian Horsey, Semantic Router superfast decision layer for LLMs and AI agents, Geeky Gadgets
- azhar, Beyond Basic Chatbots: How Semantic Router is Changing the Game, AI Insights @ Medium
- Daniel Avila, Semantic Router: Enhancing Control in LLM Conversations, CodeGPT @ Medium
- Yogendra Sisodia, Stop Chat-GPT From Going Rogue In Production With Semantic Router, Medium
- Aniket Hingane, LLM Apps: Why you Must Know Semantic Router in 2024: Part 1, Medium
- Adrien Sales, 🔀 Semantic Router w. ollama/gemma2 : real life 10ms hotline challenge 🤯
- Adrien Sales, Kaggle Notebook 🔀 Semantic Router:
ollama/gemma2:9bhotline
1 maintainer
Metadata
1 maintainer
Release files for semantic-router 0.3.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| semantic_router-0.3.2.tar.gz | 116.2 kB | Without provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| semantic_router-0.3.2-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 271.2 kB
Release files / semantic_router-0.3.2.tar.gz
| Download URL | semantic_router-0.3.2.tar.gz |
|---|---|
| Size | 116.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
26dd4d50f18539e9b77f1cc4097d87c8b34842e4742ed1d2dc850f872736f97e
|
|
BLAKE2b-256 checksum How to use checksums |
64961a9b8abaa5261267ad7cdc7ed2e163708070ca7d8f59a61a607c3c2d9446
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.13.4 {"installer":{"name":"uv","version":"0.13.4","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / semantic_router-0.3.2-py3-none-any.whl
| Download URL | semantic_router-0.3.2-py3-none-any.whl |
|---|---|
| Size | 155.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
70fffd063c3296bfb12c0b89ad632049efc9792f576a7ac79f8e28e3ce5aac47
|
|
BLAKE2b-256 checksum How to use checksums |
b91e1c64201d3c7c7cf65e44d5bf3f8a85a19b17d660df32cd92ed5756f396fd
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.13.4 {"installer":{"name":"uv","version":"0.13.4","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Security for semantic-router 0.3.2
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
