Simple vLLM/OpenAI textgen benchmark with optional GPU telemetry from TTC library
Project description
ttc_bench
Benchmark for OpenAI/vLLM /v1/chat/completions endpoints with optional local GPU telemetry.
Install
pip install ttc_bench
# env (optional)
echo 'MODEL_BASE_URL=http://host:3000/v1' > .env
echo 'MODEL_API_KEY=' >> .env
echo 'DEFAULT_MODEL=Qwen2.5-72B-Instruct-GPTQ-Int4' >> .env
# run
ttc_bench run --task textgen --steps 10 --batch 2 --seq-out 128 \
--base-url "http://host:3000/v1" --model "Qwen2.5-72B-Instruct-GPTQ-Int4"
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
ttcbench-0.1.5.tar.gz
(8.9 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
ttcbench-0.1.5-py3-none-any.whl
(11.1 kB
view details)
File details
Details for the file ttcbench-0.1.5.tar.gz.
File metadata
- Download URL: ttcbench-0.1.5.tar.gz
- Upload date:
- Size: 8.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
2046ae0d849383fb7a0f422959074ce4c5357d381d0a2e0b11959a468ea434d9
|
|
| MD5 |
9bf043f205568ca6df71f2cb667a836c
|
|
| BLAKE2b-256 |
d86218f5b290132adc01cbf85dbbb73f0faedfbda429c3144b42f62e95d6505b
|
File details
Details for the file ttcbench-0.1.5-py3-none-any.whl.
File metadata
- Download URL: ttcbench-0.1.5-py3-none-any.whl
- Upload date:
- Size: 11.1 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
1f070d26fef9a54eb9e9973c8caf164ce0a66d41fe8dfd306074fa3d6f37e60c
|
|
| MD5 |
5fed346b7d4379cc9c6d36b9976e1bbf
|
|
| BLAKE2b-256 |
1f1cc9de18e32b05c1b2d187dd5fcec2bf9eeaba50591b6a543af0560e76b5de
|
