Client for the vLLM API with minimal dependencies
Project description
vLLM Client
Overview
Client for the vLLM API with minimal dependencies.
Installation
pip install vllm-client
Examples
See example.py for the following:
- Single generation
- Streaming
- Batch inference
It should work out of the box with a vLLM API server.
Notes
sampling_params.pyneeds to be kept in sync with vLLM. It is a simplified version of their class, containing only the code required on client side.
Another programming languages
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
No source distribution files available for this release.See tutorial on generating distribution archives.
Built Distribution
File details
Details for the file vllm_client-0.2.7.1-py3-none-any.whl.
File metadata
- Download URL: vllm_client-0.2.7.1-py3-none-any.whl
- Upload date:
- Size: 9.8 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.12
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 4fca76a6c096daa76bdd49a085b0710af1127753602acc31aa6c1537ab515a53 |
|
| MD5 | 0237035d0be26ad834ed8459c6b1aa1e |
|
| BLAKE2b-256 | 70bc3144b61423bf4bd54ac2a084880c6ed10c879912516003d77ebc42f5ff4b |
