Loss of provenance

A previous release had provenance metadata, but this release does not. View details.

SMG Python Bindings

This directory contains the Python bindings for SMG (Shepherd Model Gateway), built using maturin and PyO3.

Quick Start

Installation

pip install maturin
cd smg/bindings/python
maturin develop --features vendored-openssl

Usage

The smg serve command launches backend workers and the SMG router in a single command:

# sglang with gRPC (default)
smg serve --backend sglang --model-path /path/to/model --port 8080

# sglang with HTTP
smg serve --backend sglang --model-path /path/to/model --port 8080 --connection-mode http

# vLLM (gRPC only)
smg serve --backend vllm --model /path/to/model --port 8080

# TensorRT-LLM (gRPC only)
smg serve --backend trtllm --model /path/to/model --port 8080

# Multiple workers (data parallel)
smg serve --backend sglang --model-path /path/to/model --port 8080 --dp-size 4

Serve Options

Option	Default	Description
`--backend`	`sglang`	Backend to use: `sglang`, `vllm`, or `trtllm`
`--connection-mode`	`grpc`	Connection mode: `grpc`, `http`, or `zmq`. trtllm only supports grpc; tokenspeed only zmq; `zmq` (a same-host direct engine connection) is available for vllm, tokenspeed, and sglang. Over `zmq` the router waits for the engine's startup handshake, whose READY comes only once the engine has loaded its model: the router's `--zmq-startup-timeout-secs` (`--router-zmq-startup-timeout-secs` here; default 3600) bounds how long any one stage of it may stay silent, and must cover the engine's whole init
`--host`	`127.0.0.1`	Host for the router
`--port`	`8080`	Port for the router
`--dp-size`	`1`	Data parallel size (number of worker replicas)
`--worker-host`	`127.0.0.1`	Host for worker processes
`--worker-base-port`	`31000`	Base port for workers
`--worker-startup-timeout`	`300`	Seconds to wait for workers to become healthy

Backend-specific options (e.g., --tensor-parallel-size, --quantization) are passed through to the backend.

Secrets from the environment or files

The serving keys and the store URLs need not be on the command line, where other processes can read them (/proc/<pid>/cmdline, process listings, a Deployment's args). Each is also read from an environment variable, or from a file such as a mounted Secret, by the Python launcher (smg launch, smg serve with the --router- prefix) and by the Rust binary alike:

Setting	Flag	Environment	File flag	File environment
Shared API key	`--api-key`	`SMG_API_KEY`	`--api-key-file`	`SMG_API_KEY_FILE`
Per-tenant API keys	`--tenant-api-key` (repeatable)	`SMG_TENANT_API_KEYS`	`--tenant-api-keys-file`	`SMG_TENANT_API_KEYS_FILE`
PostgreSQL URL	`--postgres-db-url`	`SMG_POSTGRES_DB_URL`	`--postgres-db-url-file`	`SMG_POSTGRES_DB_URL_FILE`
Redis URL	`--redis-url`	`SMG_REDIS_URL`	`--redis-url-file`	`SMG_REDIS_URL_FILE`
Oracle credentials	`--oracle-connect-descriptor`, `--oracle-username`, `--oracle-password`	`ATP_DSN`, `ATP_USER`, `ATP_PASSWORD`	–	–

The flag beats the environment. A value (from the flag or the environment) together with a file for the same secret is refused at start-up rather than ranked: the second source of one secret is a stale one about to be honoured.
A single-value file holds the value alone; the surrounding whitespace (a trailing newline) is dropped, and so is the whitespace around a single value from the environment (a Secret made from a file carries the editor's newline into both); the ATP_* values are the exception, see below. An empty file, or one with more than one line, is refused.
SMG_TENANT_API_KEYS holds tenant_id:key entries separated by , or a newline; the file holds one entry per line, blank lines and lines starting with # skipped. A source that is set but holds no entry is refused.
A store URL file is read for the selected --history-backend alone. The launcher still reads the older POSTGRES_DB_URL and REDIS_URL names when the SMG_ names are unset.
The Oracle credentials have no file source; the launcher and the binary (--oracle-dsn, --oracle-user, --oracle-password) read the ATP_* variables when the flags are absent, as they always did, and use them exactly as set, with no trimming: a Secret for them is made with --from-literal, or from a file that ends without a newline (a password that ends in one is a wrong password to the database).
No value is echoed: a refusal names the sources and the path, and --help shows no value of a variable. The Rust binary's start-up record (CREDENTIAL SOURCES) names each secret's source as flag, env or file; the Python launcher emits no such record.

Directory Structure

bindings/python/
├── src/                    # Source code (src layout)
│   ├── lib.rs              # Rust/PyO3 bindings implementation
│   └── smg/                # Python source code
│       ├── __init__.py
│       ├── cli.py          # CLI entry point
│       ├── serve.py        # smg serve implementation
│       ├── launch_router.py
│       ├── router.py
│       └── router_args.py
├── tests/                  # Python unit tests
├── Cargo.toml              # Rust package configuration
├── pyproject.toml          # Python package configuration
└── README.md               # This file

Building

Development Build

pip install maturin
cd smg/bindings/python
maturin develop --features vendored-openssl

Production Build

cd smg/bindings/python
maturin build --release --out dist --features vendored-openssl
pip install dist/smg-*.whl

Testing

cd smg/bindings/python
pytest tests/

Adding a router flag

A flag lives in a few places, and the tests pin them against each other:

model_gateway/src/main.rs: the clap flag, in its group of CliArgs. The Rust CLI is the reference: tests/test_cli_parity.py fails on a flag that exists on one side only.
src/smg/router_args.py: a RouterArgs field, and its option in add_cli_args under the matching help group. The field goes at its alphabetical place among the keyword-only fields (every field after worker_urls, host, port), never at the tail: tests/test_arg_parser.py pins the order and says where a misplaced field belongs.
src/lib.rs: the Router struct field, the #[pyo3(signature)] entry with the flag's default, the parameter of new and the field of the literal new builds, each at the same alphabetical place among the keyword-only parameters (every parameter after worker_urls, policy, host, port; pyo3 wants the signature and the parameters in one order), plus the line that hands the value to the router configuration in to_router_config. tests/test_router_config.py pins the order; tests/test_router_args_parity.py fails on a dataclass field the constructor does not take, and on a parameter the launcher never sets.
The flag's own tests go next to the related ones, or in a file of their own, not at the end of tests/test_arg_parser.py, where every flag's tests would meet.

The keyword-only parameters have no position, so two pull requests that each add a flag change different lines of each file and merge without a conflict.

Metadata

Release files for tokenspeed-smg 1.13.0.post20261023

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Loss of provenance

These files do not have provenance metadata, despite files in a prior release including this data.

We recommend further investigation before installing these files. View details.

Source distribution (sdist)

Source distribution for tokenspeed-smg 1.13.0.post20261023
File	Size	Uploaded	Provenance
tokenspeed_smg-1.13.0.post20261023.tar.gz	9.6 MB	Oct 23, 2026	Without provenance	Details

Built distributions (wheels)

Table of built distributions (wheels) for tokenspeed-smg 1.13.0.post20261023
File	Interpreter	ABI	Platform	Reset
tokenspeed_smg-1.13.0.post20261023-cp311-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 52.6 MB Oct 23, 2026 Without provenance	CPython 3.11	abi3	Linux glibc 2.17+ x86-64	Details
tokenspeed_smg-1.13.0.post20261023-cp311-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl 50.2 MB Oct 23, 2026 Without provenance	CPython 3.11	abi3	Linux glibc 2.17+ ARM64	Details

Total release size: 112.4 MB

Release files / tokenspeed_smg-1.13.0.post20261023.tar.gz

Loss of provenance

This file belongs to a release without provenance metadata, despite a prior release including this data.

We recommend further investigation before installing this file. View details.

Download URL	tokenspeed_smg-1.13.0.post20261023.tar.gz
Size	9.6 MB
Tags	Source
SHA-256 checksum How to use checksums	`401b36a2c4941eddfebf86e0f0f507e851c0a41e1bd8bcd2eee2064e93c04776`
BLAKE2b-256 checksum How to use checksums	`13523d6be7247f28f446d210e3a8137ea6c193fc3d5eaf867bf750b25286c11c`
Upload date	Oct 23, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/7.0.0 CPython/3.13.16`

Release files / tokenspeed_smg-1.13.0.post20261023-cp311-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

Loss of provenance

This file belongs to a release without provenance metadata, despite a prior release including this data.

We recommend further investigation before installing this file. View details.

Download URL	tokenspeed_smg-1.13.0.post20261023-cp311-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Size	52.6 MB
Tags	CPython 3.11 Linux glibc 2.17+ x86-64 abi3
SHA-256 checksum How to use checksums	`3d74e5e14ec990cf8c262afd38de9e2d69e465f5aa537a14350294458c15b49c`
BLAKE2b-256 checksum How to use checksums	`1f32b1cfc871732ed888880cfe67a0e612b2835bd458f32507f3c327875c2532`
Upload date	Oct 23, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/7.0.0 CPython/3.13.16`

Release files / tokenspeed_smg-1.13.0.post20261023-cp311-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

Loss of provenance

This file belongs to a release without provenance metadata, despite a prior release including this data.

We recommend further investigation before installing this file. View details.

Download URL	tokenspeed_smg-1.13.0.post20261023-cp311-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Size	50.2 MB
Tags	CPython 3.11 Linux glibc 2.17+ ARM64 abi3
SHA-256 checksum How to use checksums	`b49e2998192f46eed3a811d894dff28116a9481da6331ec4ac9212a800b32017`
BLAKE2b-256 checksum How to use checksums	`11488495d0ac7473a8bbdaf1249c05dd87e013047a9781a45b3ed951c6bae4c1`
Upload date	Oct 23, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/7.0.0 CPython/3.13.16`

Security for tokenspeed-smg 1.13.0.post20261023

Report project as malware

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Loss of provenance

A previous release of these files had provenance metadata, but this release does not.

While this could reflect a routine change by the project maintainers, it may also indicate that the release files have been compromised.

Release comparison

Removed 1.13.0.post20261016 3 files With provenance
Added 1.13.0.post20261023 3 files Without provenance

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.