Simple vLLM/OpenAI textgen benchmark with optional GPU telemetry
Reason this release was yanked:
Update parametres
Project description
ttc_bench
Benchmark for OpenAI/vLLM /v1/chat/completions endpoints with optional local GPU telemetry.
Install
pip install ttc_bench
# env (optional)
echo 'MODEL_BASE_URL=http://host:3000/v1' > .env
echo 'MODEL_API_KEY=' >> .env
echo 'DEFAULT_MODEL=Qwen2.5-72B-Instruct-GPTQ-Int4' >> .env
# run
ttc_bench run --task textgen --steps 10 --batch 2 --seq-out 128 \
--base-url "http://host:3000/v1" --model "Qwen2.5-72B-Instruct-GPTQ-Int4"
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
ttcbench-0.1.3.tar.gz
(7.9 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file ttcbench-0.1.3.tar.gz.
File metadata
- Download URL: ttcbench-0.1.3.tar.gz
- Upload date:
- Size: 7.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
b5c0315a88b125d7a04e0dc90e645511ef4f5558886edaa021580f882ebb2b97
|
|
| MD5 |
8cb9e313863c9b1179c91f9a18f9d359
|
|
| BLAKE2b-256 |
5428f7c2c78e96d712774e75bcc7b720c81b57f555ffcc658d416e0dce280045
|
File details
Details for the file ttcbench-0.1.3-py3-none-any.whl.
File metadata
- Download URL: ttcbench-0.1.3-py3-none-any.whl
- Upload date:
- Size: 9.1 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
5b22d9544e1a6cdad2745a312bcd8654dca7e39948558bda4fba6bcb20812d68
|
|
| MD5 |
4bee144d09d706e3fb2e54cd1367825c
|
|
| BLAKE2b-256 |
8a684a24da4398f2299021a6349776c08ca58d54bf39efc6a6ce01e5f4b1eb1c
|
