bento-sgl-router
Helper utilities for bundling an SGLang Router inside a BentoML service. It exposes a drop-in replacement for bentoml.service that starts the router sidecar, keeps it in sync with your LLM workers, and surfaces router metrics alongside your Bento.
Overview
- Wraps an existing BentoML service definition and injects an
sglang-routerprocess. - Automatically syncs worker hosts discovered from your
llmdependency during readiness probes. - Exposes router Prometheus metrics by merging them into the Bento service metrics endpoint.
Installation
pip install bento-sgl-router
Python 3.10 or newer is required. The package installs bentoml>=1.4.25 and sglang-router>=0.1.9 as dependencies.
Quickstart
Decorate an existing BentoML LLM service to bundle it with an SGLang Router:
import bentoml
from bento_sgl_router import service
@service(name="chat-router")
class ChatLLM:
...
Any keyword arguments supported by bentoml.service (such as image, labels, or resource configs) can be passed through the decorator.
Router configuration
Control router behaviour via environment variables:
| Variable | Default | Description |
|---|---|---|
ROUTER_POLICY |
cache_aware |
Scheduling policy passed to sglang_router.launch_router. |
Set the variable in your deployment environment or via envs=[...] when declaring the service.
Development
pdm install
pdm run pytest
The project keeps the code in bento_sgl_router.py and lightweight tests under tests/.
1 maintainer
Metadata
1 maintainer
Release files for bento-sgl-router 0.0.6
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| bento_sgl_router-0.0.6.tar.gz | 2.8 kB | With provenance | Details for bento_sgl_router-0.0.6.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| bento_sgl_router-0.0.6-py3-none-any.whl | Python 3 | none | any | Details for bento_sgl_router-0.0.6-py3-none-any.whl |
Total release size: 6.3 kB
Release files / bento_sgl_router-0.0.6.tar.gz
| Download URL | bento_sgl_router-0.0.6.tar.gz |
|---|---|
| Size | 2.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
e2b1769598c46db3bfaef8725560820b9f07d0613e7628e09aa458a9ce2fee7b
|
|
BLAKE2b-256 checksum How to use checksums |
b4fe53e77d2df4ab545bfb03648f8578d82627490584709211b1928b198368b7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 28, 2025.
Transparency logRelease files / bento_sgl_router-0.0.6-py3-none-any.whl
| Download URL | bento_sgl_router-0.0.6-py3-none-any.whl |
|---|---|
| Size | 3.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
9770552f21323c36881e8057426da5480ebd471d51099a80bc0855d8b936c0d1
|
|
BLAKE2b-256 checksum How to use checksums |
dfb5493131f0ec237c6451e8e21dc840ea12936c0fb25d8969115f62f5088520
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 28, 2025.
Transparency logSecurity for bento-sgl-router 0.0.6
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
All 2 files in this release have been uploaded with provenance metadata.
Attestation for bento_sgl_router-0.0.6.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 28, 2025.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
