Simple vLLM/OpenAI textgen benchmark with optional GPU telemetry
Reason this release was yanked:
Update parametres
Project description
ttc_bench
Benchmark for OpenAI/vLLM /v1/chat/completions endpoints with optional local GPU telemetry.
Install
pip install ttc_bench
# env (optional)
echo 'MODEL_BASE_URL=http://host:3000/v1' > .env
echo 'MODEL_API_KEY=' >> .env
echo 'DEFAULT_MODEL=Qwen2.5-72B-Instruct-GPTQ-Int4' >> .env
# run
ttc_bench run --task textgen --steps 10 --batch 2 --seq-out 128 \
--base-url "http://host:3000/v1" --model "Qwen2.5-72B-Instruct-GPTQ-Int4"
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
ttcbench-0.1.2.tar.gz
(7.5 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file ttcbench-0.1.2.tar.gz.
File metadata
- Download URL: ttcbench-0.1.2.tar.gz
- Upload date:
- Size: 7.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
2c545c75b1e2fe725f4b51f54786c761542357c065fddd0fc891d652fe6b6cdd
|
|
| MD5 |
3408bd7f65b9f60ea611a714a66e72b2
|
|
| BLAKE2b-256 |
2961d5b46e84a6077292dc0e1d3f9be24a482eeec46f7617944016c3a6e6ac21
|
File details
Details for the file ttcbench-0.1.2-py3-none-any.whl.
File metadata
- Download URL: ttcbench-0.1.2-py3-none-any.whl
- Upload date:
- Size: 8.6 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
82452eb25d71ff60d67df299fbc46fad488824e73460a3b9dad75983b503700b
|
|
| MD5 |
b28311213d9b8679ec154bc24d942ae3
|
|
| BLAKE2b-256 |
b01ee5d9f2b85e9d3a5c8d95c40e216099149611ca3fda9dfbbb058f8c5f583f
|
