Simple vLLM/OpenAI textgen benchmark with optional GPU telemetry from TTC library
Reason this release was yanked:
Update view users
Project description
ttc_bench
Benchmark for OpenAI/vLLM /v1/chat/completions endpoints with optional local GPU telemetry.
Install
pip install ttc_bench
# env (optional)
echo 'MODEL_BASE_URL=http://host:3000/v1' > .env
echo 'MODEL_API_KEY=' >> .env
echo 'DEFAULT_MODEL=Qwen2.5-72B-Instruct-GPTQ-Int4' >> .env
# run
ttc_bench run --task textgen --steps 10 --batch 2 --seq-out 128 \
--base-url "http://host:3000/v1" --model "Qwen2.5-72B-Instruct-GPTQ-Int4"
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
ttcbench-0.1.4.tar.gz
(8.8 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
ttcbench-0.1.4-py3-none-any.whl
(10.8 kB
view details)
File details
Details for the file ttcbench-0.1.4.tar.gz.
File metadata
- Download URL: ttcbench-0.1.4.tar.gz
- Upload date:
- Size: 8.8 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
668de988e6e9304657a48db876d93c633c619c304a7c0ba017768d4cc3d265c9
|
|
| MD5 |
51dfdc89ed372f44abf87046ace8241b
|
|
| BLAKE2b-256 |
d15e75aeaf660e59b8341a1e4ca0c44fc0006cae0c742cc7775c4047f1423f40
|
File details
Details for the file ttcbench-0.1.4-py3-none-any.whl.
File metadata
- Download URL: ttcbench-0.1.4-py3-none-any.whl
- Upload date:
- Size: 10.8 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
378316c435b58b6d937ad08061ac188212c11530f0b4a2427f1a531f15cc7cf1
|
|
| MD5 |
a242529f9599ef761c7fc5c01d60ff94
|
|
| BLAKE2b-256 |
65f01af92e9f3a4500b94f184f6a44aa9aac0349bce8256b1f0ce12de42ec5f9
|
