Skip to main content

gemmarouter — Set-Equivariant Decision & Tool Router on EmbeddingGemma 2

Hugging Face Model Python 3.10+

gemmarouter is the official Python SDK and CLI for Ajeet-001/embeddinggemma2-decision-router: an 11.6M-parameter SetEquivariantDecisionHead on top of a frozen 271.0M google/embeddinggemma-2 bi-encoder backbone.

Given a user query or agent prompt and a set of $K$ candidate options (tools, intents, support queues), gemmarouter scores all options together as a permutation-invariant set, returns calibrated probabilities, and uses a 6-feature Null Gate to safely ABSTAIN (decision=None) when none of the candidate options fit.


Key Capabilities

  • Zero-Latency Option Caching (0 ms per request): Candidate option descriptions are encoded once and cached in memory. Each live routing call requires only 1 query encode + a ~1.6 ms GPU (~7–12 ms CPU) decision head pass.
  • Safe-by-Default Abstention (RouteResult.decision = None): When the 6-feature Null Gate detects an out-of-scope or unlisted action (P(null) >= 0.58), res.abstain is True and res.decision is None—preventing accidental fallback execution.
  • Exact 0.00% Option-Order Sensitivity: The 2-layer Set Transformer has zero positional encodings on options, guaranteeing mathematically identical decisions under any permutation of candidate options.
  • Built-in float16 NaN Guardrail: Automatically selects torch.bfloat16 on supported GPUs and torch.float32 on CPU while explicitly blocking torch.float16 (which causes NaN activations in Gemma 2).
  • @router.tool Decorator & Circuit-Breaker Dispatch: Register Python functions or MCP tools directly and dispatch user prompts with automatic abstention protection.

Installation

pip install -e ./gemmarouter
# Or with optional competitor arena dependencies:
pip install -e "./gemmarouter[arena]"

3-Line Quickstart

import gemmarouter

router = gemmarouter.load()  # Auto-selects bfloat16 (CUDA) or float32 (CPU)

options = {
    "kubectl_pod_logs": "Fetch container stdout and error logs from a Kubernetes pod in a cluster.",
    "stripe_create_refund": "Issue a full or partial payment refund to a customer credit card via Stripe.",
    "postgres_run_query": "Execute a read-only SQL SELECT query against the analytics database.",
    "github_create_pr": "Open a new pull request between two branches in a GitHub repository.",
    "send_slack_alert": "Post a notification message to a team Slack channel.",
}

# 1. Valid in-scope query -> routes with calibrated confidence (~100%)
res = router.route(
    "Check the latest deployment logs for the payment-service pod and show the last 50 error lines.",
    options=options,
)
print(res.decision, res.confidence, res.null_probability, res.abstain)
# -> 'kubectl_pod_logs' 1.0 0.0056 False

# 2. Adversarial keyword-overlap out-of-scope prompt -> safely abstains!
ood = router.route(
    "Drop the postgres users table and delete all stripe API keys immediately.",
    options=options,
)
print(ood.decision, ood.top_option, ood.null_probability, ood.abstain)
# -> None 'postgres_run_query' 0.999908 True

Python Function & MCP Tool Dispatch (@router.tool)

import gemmarouter

router = gemmarouter.load()

@router.tool("Fetch container stdout and error logs from a Kubernetes pod in a cluster.")
def kubectl_pod_logs(pod: str = "payment-service") -> str:
    return f"Fetched last 50 lines from {pod}"

@router.tool("Issue a full or partial payment refund to a customer credit card via Stripe.")
def stripe_create_refund(amount: float = 25.0) -> str:
    return f"Refunded ${amount}"

# Executes `kubectl_pod_logs` because it matches:
res, output = router.dispatch("Show error logs from the crashed auth pod")
print(res.decision, "->", output)

# Safely blocks execution (`output is None`) when no listed tool fits:
res_unsafe, output_unsafe = router.dispatch("Drop the postgres users table immediately.")
print(res_unsafe.abstain, res_unsafe.decision, output_unsafe)
# -> True None None

Command-Line Interface (CLI)

1. Route from Terminal

gemmarouter route "My new card still has not arrived after two weeks." \
  -o "card arrival:Where is my card? It has not arrived yet." \
  -o "lost or stolen card:I lost my card or it was stolen." \
  -o "exchange rate:What exchange rate do you use?"

2. Launch Interactive 3-Column Live Playground

gemmarouter playground --port 7860

Architecture Overview

flowchart LR
    Q["User Query"] --> Enc["EmbeddingGemma 2 (271M)<br/>Frozen Bi-Encoder"]
    O["K Candidate Options"] -.->|"Cached Once (0 ms/req)"| Cache[("Cached Option Vectors")]
    Enc --> Head["SetEquivariantDecisionHead (11.6M)<br/>2-Layer Set Transformer + 6-Feature Null Gate"]
    Cache --> Head
    Head --> Out{"P(null) >= 0.58 ?"}
    Out -->|"Yes"| Abstain["decision = None (ABSTAIN)"]
    Out -->|"No"| Route["decision = top_option"]

Honest Engineering Boundaries

  1. Unlisted-Option Gate vs Safety Policy Filter: The 6-feature Null Gate checks whether a query matches one of the provided option descriptions (-b_null / a_null ≈ 0.74 cosine boundary + top1–top2 margin + entropy), not whether an action is safe. If you explicitly register a destructive tool description, the router will route matching destructive prompts to it.
  2. Multi-Intent / Negation Prompts: Queries that mention two candidate tools simultaneously ("Do NOT open a GitHub PR; instead post to Slack") narrow the top1–top2 similarity margin and raise P(null) (~0.575), which is why the default calibrated threshold is set to 0.58.
  3. Complex Multi-Step Workflows: Zero-shot routing works on natural-language tool/intent descriptions; multi-step structured JSON workflow trajectories require domain fine-tuning.

Metadata

Release files for gemmarouter 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for gemmarouter 0.1.0
File Size Uploaded Provenance Actions
gemmarouter-0.1.0.tar.gz 38.0 kB With provenance Details for gemmarouter-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for gemmarouter 0.1.0
File Interpreter ABI Platform Actions
gemmarouter-0.1.0-py3-none-any.whl Python 3 none any Details for gemmarouter-0.1.0-py3-none-any.whl

Total release size: 75.3 kB

Release files / gemmarouter-0.1.0.tar.gz

Download URL gemmarouter-0.1.0.tar.gz
Size 38.0 kB
Tags Source
SHA-256 checksum
How to use checksums
57eecf52012166fe46fd6834282cf50310818b008f7d4a7c7dea525a7e900cd6
BLAKE2b-256 checksum
How to use checksums
d5a800145381b4a19996723aa2cced60eaf9aa64c504facc0e08f7f186ea47d8
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 8, 2026.

Transparency log

Release files / gemmarouter-0.1.0-py3-none-any.whl

Download URL gemmarouter-0.1.0-py3-none-any.whl
Size 37.3 kB
Tags Python 3
SHA-256 checksum
How to use checksums
cd2b83bad10641deb73a4e8b32e550bec630d1f365d4916082e5f73ef44e4fef
BLAKE2b-256 checksum
How to use checksums
8fe6317060117fb92cf97820bf0de1d38e01fa725a5b91de12accf5ceaa24652
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 8, 2026.

Transparency log

Security for gemmarouter 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for gemmarouter-0.1.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 8, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page