Model inference performance stress test
evalscope-perf help
evalscope-perf --help
Usage: evalscope-perf [OPTIONS] URL MODEL DATASET_PATH
╭─ Arguments ──────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ * url TEXT OpenAI URL [default: None] [required]
│ * model TEXT 模型名称 [default: None] [required]
│ * dataset_path TEXT 数据集路径 [default: None] [required]
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --max-prompt-length INTEGER 最大提示长度 [default: 256]
│ --stop TEXT 停止标记 [default: <|im_end|>]
│ --read-timeout INTEGER 读取超时 [default: 30]
│ --parallels INTEGER 并行数 [default: 1]
│ --n INTEGER 请求数 [default: 1]
│ --install-completion Install completion for the current shell.
│ --show-completion Show completion for the current shell, to copy it or customize the installation.
│ --help Show this message and exit.
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Example
Nvidia T4X4 - XInference(vLLM) - Qwen1.5-7B-Chat
evalscope-perf http://172.16.33.66:9997/v1/chat/completions gpt-4-32k \
./datasets/open_qa.jsonl \
--parallels 4 \
--parallels 8 \
--parallels 16 \
--n 20
Ascend 910B4X8 - MindIE - DeepSeek-Coder-6.7B-Instruct
evalscope-perf http://127.0.0.1:1025/v1/chat/completions qwen \
./datasets/Codefuse-Evol-Instruct-Clean-data.jsonl \
--parallels 32 \
--parallels 64 \
--parallels 100 \
--parallels 128 \
--parallels 150 \
--parallels 200 \
--parallels 256 \
--parallels 300 \
--parallels 400 \
--parallels 500 \
--parallels 600 \
--parallels 700 \
--parallels 800 \
--parallels 900 \
--parallels 1000 \
--n 2000
evalscope-perf http://127.0.0.1:11434/v1/chat/completions qwen2.5:7b
./datasets/open_qa.jsonl
--parallels 1
--parallels 2
--n 4
Metadata
Release files for evalscope-perf 1.0.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| evalscope_perf-1.0.0.tar.gz | 21.1 MB | Without provenance | Details for evalscope_perf-1.0.0.tar.gz |
Release files / evalscope_perf-1.0.0.tar.gz
| Download URL | evalscope_perf-1.0.0.tar.gz |
|---|---|
| Size | 21.1 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
c04cbf4f5985f5b3b0bedc22a1e82ccd7a864180cf4c9d081628d424660965b3
|
|
BLAKE2b-256 checksum How to use checksums |
804b0d944b37437aa4bc16043aff735337b914c6fb23679620d8b073f9765e27
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.10.9
|
Security for evalscope-perf 1.0.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.