vllm-tgis-adapter

vLLM adapter for a TGIS-compatible grpc server.

Install

vllm-tgis-adapter is available on PyPi

pip install vllm-tgis-adapter
python -m vllm_tgis_adapter

HealthCheck CLI

Installing the adapter also install a grpc healthcheck cli that can be used to monitor the status of the grpc server:

$ grpc_healtheck
health check...status: SERVING

See usage with

grpc_healthcheck --help

Build

python -m build
pip install dist/*whl
python -m vllm_tgis_adapter

Inference

This will start serving a grpc server on port 8033. This can be queried with grpcurl:

bash examples/inference.sh

Docker

Image available at quay.io/opendatahub/vllm, built from opendatahub-io/vllm's Dockerfile.ubi

docker pull quay.io/opendatahub/vllm

Inference

See examples

Contributing

Set up pre-commit for linting/style/misc fixes:

pip install pre-commit
pre-commit install
# to run on all files
pre-commit run --all-files

This project uses nox to manage test automation and uv for venv management:

pip install nox uv
nox --list  # list available sessions
nox -s tests-3.10 # run tests session for a specific python version
nox -s build-3.11 # build the wheel package
nox -s lint-3.11 -- --mypy # run linting with type checks

Testing without a GPU

The standard vllm built requires an Nvidia GPU. When this is not available, it is possible to compile vllm from source with CPU support:

git clone https://github.com/vllm-project/vllm
cd vllm

uv venv
source .venv/bin/activate

export UV_EXTRA_INDEX_URL=https://download.pytorch.org/whl/cpu \
    UV_INDEX_STRATEGY=unsafe-best-match\

.github/scripts/install_vllm_build_deps.py pyproject.toml

env \
    VLLM_TARGET_DEVICE=cpu \
    python setup.py bdist_wheel
export VLLM_VERSION_OVERRIDE=$PWD/dist/*whl
# the nox session can now be run with the custom built vllm cpu version

making it possible to run the tests on most hardware. Please note that the uv extra index url is required in order to install the torch CPU version.

Metadata

Release files for vllm-tgis-adapter 0.10.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for vllm-tgis-adapter 0.10.0
File	Size	Uploaded
vllm_tgis_adapter-0.10.0.tar.gz	92.7 kB	Dec 17, 2025	Details

Built distribution (wheel)

Table of built distributions (wheels) for vllm-tgis-adapter 0.10.0
File	Interpreter	ABI	Platform	Reset
vllm_tgis_adapter-0.10.0-py3-none-any.whl 51.8 kB Dec 17, 2025	Python 3	none	any	Details

Total release size: 144.5 kB

Release files / vllm_tgis_adapter-0.10.0.tar.gz

Download URL	vllm_tgis_adapter-0.10.0.tar.gz
Size	92.7 kB
Tags	Source
SHA-256 checksum How to use checksums	`7da42878a8bf43e1141bcaaf313f9f8e2b25f99d244bc20b86463c88a46f308a`
BLAKE2b-256 checksum How to use checksums	`738101fe1ffc096d971beb64fc170b96e1e2cf6ad01757f77b83a243f6c291d2`
Upload date	Dec 17, 2025
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/6.1.0 CPython/3.13.7`

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/opendatahub-io/vllm-tgis-adapter
Publishing commit: github.com/opendatahub-io/vllm-tgis-adapter/tree/df3596ae59854c7e7efaf9fee3c3ace9623c95ee

Workflow

Publishing configuration: github.com/opendatahub-io/vllm-tgis-adapter/blob/df3596ae59854c7e7efaf9fee3c3ace9623c95ee/.github/workflows/release.yaml
Publishing logs: github.com/opendatahub-io/vllm-tgis-adapter/actions/runs/20310464152/attempts/1

Artifact

Subject: vllm_tgis_adapter-0.10.0.tar.gz
SHA-256 checksum: 7da42878a8bf43e1141bcaaf313f9f8e2b25f99d244bc20b86463c88a46f308a Verifying attestations

Release files / vllm_tgis_adapter-0.10.0-py3-none-any.whl

Download URL	vllm_tgis_adapter-0.10.0-py3-none-any.whl
Size	51.8 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`7f97b4dfc0cfaaa23a9d662ca47b4c0db1dc48c275983c60d719afeb5f1d635b`
BLAKE2b-256 checksum How to use checksums	`18094b3ce86da62bd70d52e2e5cb28816b3f705d4de79ec2dc7057f74563ed18`
Upload date	Dec 17, 2025
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/6.1.0 CPython/3.13.7`

Provenance

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/opendatahub-io/vllm-tgis-adapter
Publishing commit: github.com/opendatahub-io/vllm-tgis-adapter/tree/df3596ae59854c7e7efaf9fee3c3ace9623c95ee

Workflow

Publishing configuration: github.com/opendatahub-io/vllm-tgis-adapter/blob/df3596ae59854c7e7efaf9fee3c3ace9623c95ee/.github/workflows/release.yaml
Publishing logs: github.com/opendatahub-io/vllm-tgis-adapter/actions/runs/20310464152/attempts/1

Artifact

Subject: vllm_tgis_adapter-0.10.0-py3-none-any.whl
SHA-256 checksum: 7f97b4dfc0cfaaa23a9d662ca47b4c0db1dc48c275983c60d719afeb5f1d635b Verifying attestations