smg-grpc-servicer
gRPC servicer implementations for LLM inference engines. Supports vLLM, MLX, TokenSpeed, and SGLang.
Installation
For vLLM:
pip install smg-grpc-servicer[vllm]
For MLX:
pip install smg-grpc-servicer[mlx]
For TokenSpeed, install the TokenSpeed runtime first, then install the servicer bridge:
pip install smg-grpc-servicer
For SGLang:
pip install smg-grpc-servicer[sglang]
Usage
vLLM
vllm serve meta-llama/Llama-2-7b-hf --grpc
MLX
python -m smg_grpc_servicer.mlx --model meta-llama/Llama-2-7b-hf --host 0.0.0.0 --port 50051
TokenSpeed
python -m smg_grpc_servicer.tokenspeed --model meta-llama/Llama-2-7b-hf --host 0.0.0.0 --port 50051
SGLang
sglang serve --model-path meta-llama/Llama-2-7b-hf --grpc-mode
Architecture
smg-grpc-servicer[vllm] ──optional dep──> vllm (lazy import)
smg-grpc-servicer[mlx] ──optional dep──> mlx-lm (lazy import)
smg-grpc-servicer ──external runtime──> tokenspeed (lazy import)
smg-grpc-servicer[sglang] ──optional dep──> sglang (lazy import)
smg-grpc-servicer ──depends on────> smg-grpc-proto (hard dependency)
vllm ──optional──────> smg-grpc-servicer (via vllm serve --grpc)
sglang ──optional──────> smg-grpc-servicer (via --grpc-mode)
Backend dependencies are isolated via extras or runtime installs to avoid conflicts between vLLM, MLX, TokenSpeed, and SGLang.
Development
See DEVELOPMENT.md for local development setup, CI, and release workflows.
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file tokenspeed_smg_grpc_servicer-0.8.0.post20260815.tar.gz.
File metadata
- Download URL: tokenspeed_smg_grpc_servicer-0.8.0.post20260815.tar.gz
- Upload date:
- Size: 109.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
22d57bd0383815ee48f4b8336ea01a102d6fde2908b96b05254bc2164f90569e
|
|
| MD5 |
20e3d83b0669f71dc0d8d640a4405bef
|
|
| BLAKE2b-256 |
f48eea2e083979e11d4f1d2f3c7e5c9e586bfdc03746e774c615d4d032374d7a
|
Provenance
The following attestation bundles were made for tokenspeed_smg_grpc_servicer-0.8.0.post20260815.tar.gz:
Publisher:
tokenspeed-smg-grpc-servicer.yml on lightseekorg/tokenspeed-third-party
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
tokenspeed_smg_grpc_servicer-0.8.0.post20260815.tar.gz -
Subject digest:
22d57bd0383815ee48f4b8336ea01a102d6fde2908b96b05254bc2164f90569e - Sigstore transparency entry: 2468768356
- Sigstore integration time:
-
Permalink:
lightseekorg/tokenspeed-third-party@4b799992b1959bffed4b4540cd71fe09f83ea401 -
Branch / Tag:
refs/heads/main - Owner: https://github.com/lightseekorg
-
Access:
private
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
tokenspeed-smg-grpc-servicer.yml@4b799992b1959bffed4b4540cd71fe09f83ea401 -
Trigger Event:
workflow_dispatch
-
Statement type:
File details
Details for the file tokenspeed_smg_grpc_servicer-0.8.0.post20260815-py3-none-any.whl.
File metadata
- Download URL: tokenspeed_smg_grpc_servicer-0.8.0.post20260815-py3-none-any.whl
- Upload date:
- Size: 113.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
9a82cf846bd87d39a9b27ab6f01e7ef985d032ebfdf2e8dc56dc8fec92395499
|
|
| MD5 |
4ab90cdd6315211c9c57252638db26e7
|
|
| BLAKE2b-256 |
8645effbcfa17b2b7ead878834f3d0378bf724deec2d619571f12f31ffd2bcd1
|
Provenance
The following attestation bundles were made for tokenspeed_smg_grpc_servicer-0.8.0.post20260815-py3-none-any.whl:
Publisher:
tokenspeed-smg-grpc-servicer.yml on lightseekorg/tokenspeed-third-party
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
tokenspeed_smg_grpc_servicer-0.8.0.post20260815-py3-none-any.whl -
Subject digest:
9a82cf846bd87d39a9b27ab6f01e7ef985d032ebfdf2e8dc56dc8fec92395499 - Sigstore transparency entry: 2468768372
- Sigstore integration time:
-
Permalink:
lightseekorg/tokenspeed-third-party@4b799992b1959bffed4b4540cd71fe09f83ea401 -
Branch / Tag:
refs/heads/main - Owner: https://github.com/lightseekorg
-
Access:
private
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
tokenspeed-smg-grpc-servicer.yml@4b799992b1959bffed4b4540cd71fe09f83ea401 -
Trigger Event:
workflow_dispatch
-
Statement type:
