autoinference-utils
Shared endpoint abstractions (SGLangEndpoint, VLLMEndpoint) for autoinference deployments.
For SGLang models whose custom Transformers code cannot load from symlinked
Hugging Face snapshots, set AUTOINFERENCE_MATERIALIZE_MODEL_PATH=1. The endpoint
copies local base/draft model code and metadata into unique temporary directories
and symlinks top-level safetensors weights. Repository IDs pass through unchanged.
Leave the setting unset to preserve the original paths.
Publishing
Bump version in pyproject.toml and merge to main — CI publishes automatically via trusted publishing.
To publish manually: cd autoinference_utils && uv build && uv publish
1 maintainer
Metadata
1 maintainer
Release files for autoinference-utils 0.2.7
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| autoinference_utils-0.2.7.tar.gz | 12.9 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| autoinference_utils-0.2.7-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 25.0 kB
Release files / autoinference_utils-0.2.7.tar.gz
| Download URL | autoinference_utils-0.2.7.tar.gz |
|---|---|
| Size | 12.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
fe674c5cee6d15a3d4d8a98aebb75290aef344f5f0807647a14b8c5ff49696bb
|
|
BLAKE2b-256 checksum How to use checksums |
26a6ccf9607cd1b4619157012f52d0f200f6173b1619048de766e1dcd4f6ab0a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.23 {"installer":{"name":"uv","version":"0.12.23","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / autoinference_utils-0.2.7-py3-none-any.whl
| Download URL | autoinference_utils-0.2.7-py3-none-any.whl |
|---|---|
| Size | 12.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
ac4f1b9a4647729cafa81b8afbe8595cbd518a556200b6755856159fc9b7fe6f
|
|
BLAKE2b-256 checksum How to use checksums |
f9f78a9be7c7e210e75cca18c0fc9eafcd33879ab06184e5611365fd56e1be07
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.23 {"installer":{"name":"uv","version":"0.12.23","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
