📚 Documentation : https://france-travail.github.io/happy_vllm/
happy_vLLM is a REST API for vLLM which was developed with production in mind. It adds some functionalities to vLLM.
Installation
You can install happy_vLLM using pip:
pip install happy_vllm
Or build it from source:
git clone https://github.com/France-Travail/happy_vllm.git
cd happy_vllm
pip install -e .
Quickstart
Just use the entrypoint happy-vllm (see arguments for a list of all possible arguments)
happy-vllm --model path_to_model --host 127.0.0.1 --port 5000 --model-name my_model
It will launch the API and you can directly query it for example with
curl 127.0.0.1:5000/v1/info
To get various information on the application or
curl 127.0.0.1:5000/v1/completions -d '{"prompt": "Hey,", "model": "my_model"}'
if you want to generate your first LLM response using happy_vLLM. See endpoints for more details on all the endpoints provided by happy_vLLM.
Deploy with Docker image
A docker image is available from the Github Container Registry :
docker pull ghcr.io/france-travail/happy_vllm:latest
See deploying_with_docker for more details on how to serve happy_vLLM with docker.
Swagger
You can reach the swagger UI at the /docs endpoint (so for example by default at 127.0.0.1:5000/docs). You will be displayed all the endpoints and examples on how to use them.
1 maintainer
Metadata
1 maintainer
Release files for happy-vllm 1.2.5
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| happy_vllm-1.2.5.tar.gz | 62.8 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| happy_vllm-1.2.5-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 123.9 kB
Release files / happy_vllm-1.2.5.tar.gz
| Download URL | happy_vllm-1.2.5.tar.gz |
|---|---|
| Size | 62.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
2241b08fa28d2e3428ed7068161e6e6cfcc89225aa55616a476d535f2b050f2d
|
|
BLAKE2b-256 checksum How to use checksums |
19fb44e5dc4996e4dd0c294cfcd9ebfa68c4cfc170b4200f193555d9120a6ac5
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.8.20
|
Release files / happy_vllm-1.2.5-py3-none-any.whl
| Download URL | happy_vllm-1.2.5-py3-none-any.whl |
|---|---|
| Size | 61.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
5149c41f3131226c1e50f7519abef6b487b110cbb69edb6ac6190fd79b87f03a
|
|
BLAKE2b-256 checksum How to use checksums |
f02fa86dd4932c018e46ec6a5373f0a49b347d965398105ef037dba73cda5775
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.8.20
|
