superduper_vllm

Superduper allows users to work with self-hosted LLM models via vLLM.

Installation

pip install superduper_vllm

API

Class	Description
`superduper_vllm.model.VllmChat`	VLLM model for chatting.
`superduper_vllm.model.VllmCompletion`	VLLM model for generating completions.

Examples

VllmChat

from superduper_vllm import VllmChat
vllm_params = dict(
    model="hugging-quants/Meta-Llama-3.1-8B-Instruct-AWQ-INT4",
    quantization="awq",
    dtype="auto",
    max_model_len=1024,
    tensor_parallel_size=1,
)
model = VllmChat(identifier="model", vllm_params=vllm_params)
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "hello"},
]

Chat with chat format messages

model.predict(messages)

Chat with text format messages

model.predict("hello")

VllmCompletion

from superduper_vllm import VllmCompletion
vllm_params = dict(
    model="hugging-quants/Meta-Llama-3.1-8B-Instruct-AWQ-INT4",
    quantization="awq",
    dtype="auto",
    max_model_len=1024,
    tensor_parallel_size=1,
)
model = VllmCompletion(identifier="model", vllm_params=vllm_params)
model.predict("hello")

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

superduper_vllm-0.10.0.tar.gz (13.9 kB view details)

Uploaded Aug 26, 2025 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

superduper_vllm-0.10.0-py3-none-any.whl (12.6 kB view details)

Uploaded Aug 26, 2025 Python 3

File details

Details for the file superduper_vllm-0.10.0.tar.gz.

File metadata

Download URL: superduper_vllm-0.10.0.tar.gz
Upload date: Aug 26, 2025
Size: 13.9 kB
Tags: Source
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for superduper_vllm-0.10.0.tar.gz
Algorithm	Hash digest
SHA256	`48f58642d3077944584d91837d41b9a054c571da33d4b6db30c6988958ea50f7`
MD5	`e2d24019bb97293922ee25b684eb7f65`
BLAKE2b-256	`61c53a461e6a2309407fc43532cbcccdac85a48759dec6843c8beab449ad56b7`

See more details on using hashes here.

Provenance

The following attestation bundles were made for superduper_vllm-0.10.0.tar.gz:

Publisher: release_plugins.yaml on superduper-io/superduper

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: superduper_vllm-0.10.0.tar.gz
- Subject digest: 48f58642d3077944584d91837d41b9a054c571da33d4b6db30c6988958ea50f7
- Sigstore transparency entry: 433509231
- Sigstore integration time: Aug 26, 2025
Source repository:
- Permalink: superduper-io/superduper@24c447e4aeb52d5a0e39bab39128de8abfee652c
- Branch / Tag: refs/heads/0.10
- Owner: https://github.com/superduper-io
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: release_plugins.yaml@24c447e4aeb52d5a0e39bab39128de8abfee652c
- Trigger Event: push