smg-grpc-servicer

gRPC servicer implementations for LLM inference engines. Supports vLLM, MLX, TokenSpeed, and SGLang.

Installation

For vLLM:

pip install smg-grpc-servicer[vllm]

For MLX:

pip install smg-grpc-servicer[mlx]

For TokenSpeed, install the TokenSpeed runtime first, then install the servicer bridge:

pip install smg-grpc-servicer

For SGLang:

pip install smg-grpc-servicer[sglang]

Usage

vLLM

vllm serve meta-llama/Llama-2-7b-hf --grpc

MLX

python -m smg_grpc_servicer.mlx --model meta-llama/Llama-2-7b-hf --host 0.0.0.0 --port 50051

TokenSpeed

python -m smg_grpc_servicer.tokenspeed --model meta-llama/Llama-2-7b-hf --host 0.0.0.0 --port 50051

SGLang

sglang serve --model-path meta-llama/Llama-2-7b-hf --grpc-mode

Architecture

smg-grpc-servicer[vllm]    ──optional dep──>  vllm       (lazy import)
smg-grpc-servicer[mlx]     ──optional dep──>  mlx-lm     (lazy import)
smg-grpc-servicer          ──external runtime──>  tokenspeed (lazy import)
smg-grpc-servicer[sglang]  ──optional dep──>  sglang     (lazy import)
smg-grpc-servicer          ──depends on────>  smg-grpc-proto  (hard dependency)
vllm                       ──optional──────>  smg-grpc-servicer (via vllm serve --grpc)
sglang                     ──optional──────>  smg-grpc-servicer (via --grpc-mode)

Backend dependencies are isolated via extras or runtime installs to avoid conflicts between vLLM, MLX, TokenSpeed, and SGLang.

Development

See DEVELOPMENT.md for local development setup, CI, and release workflows.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tokenspeed_smg_grpc_servicer-0.8.0.post20260820.tar.gz (109.9 kB view details)

Uploaded Aug 20, 2026 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

tokenspeed_smg_grpc_servicer-0.8.0.post20260820-py3-none-any.whl (113.5 kB view details)

Uploaded Aug 20, 2026 Python 3

File details

Details for the file tokenspeed_smg_grpc_servicer-0.8.0.post20260820.tar.gz.

File metadata

Download URL: tokenspeed_smg_grpc_servicer-0.8.0.post20260820.tar.gz
Upload date: Aug 20, 2026
Size: 109.9 kB
Tags: Source
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for tokenspeed_smg_grpc_servicer-0.8.0.post20260820.tar.gz
Algorithm	Hash digest
SHA256	`85ebee068179a154dca7e4037957e82bfa317937d63369fca1108fa66be52893`
MD5	`59930af8aae09f56c047fde8c514eb97`
BLAKE2b-256	`4ed31903946cb31441283513976e8ddb801ab9b9a51d26d5dc9284f42ef8f389`

See more details on using hashes here.

Provenance

The following attestation bundles were made for tokenspeed_smg_grpc_servicer-0.8.0.post20260820.tar.gz:

Publisher: tokenspeed-smg-grpc-servicer.yml on lightseekorg/tokenspeed-third-party

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: tokenspeed_smg_grpc_servicer-0.8.0.post20260820.tar.gz
- Subject digest: 85ebee068179a154dca7e4037957e82bfa317937d63369fca1108fa66be52893
- Sigstore transparency entry: 2537297728
- Sigstore integration time: Aug 20, 2026
Source repository:
- Permalink: lightseekorg/tokenspeed-third-party@b607bfb47e3b8d4102276681f3bce1ccf1ff77ca
- Branch / Tag: refs/heads/main
- Owner: https://github.com/lightseekorg
- Access: private
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: tokenspeed-smg-grpc-servicer.yml@b607bfb47e3b8d4102276681f3bce1ccf1ff77ca
- Trigger Event: workflow_dispatch