Skip to main content

Model inference performance stress test

evalscope-perf help

evalscope-perf --help
 Usage: evalscope-perf [OPTIONS] URL MODEL DATASET_PATH                                                                     
                                                                                                                            
╭─ Arguments ──────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ *    url               TEXT  OpenAI URL [default: None] [required]
│ *    model             TEXT  模型名称 [default: None] [required]
│ *    dataset_path      TEXT  数据集路径 [default: None] [required]
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --max-prompt-length         INTEGER  最大提示长度 [default: 256]
│ --stop                      TEXT     停止标记 [default: <|im_end|>]
│ --read-timeout              INTEGER  读取超时 [default: 30]
│ --parallels                 INTEGER  并行数 [default: 1]
│ --n                         INTEGER  请求数 [default: 1]
│ --install-completion                 Install completion for the current shell.
│ --show-completion                    Show completion for the current shell, to copy it or customize the installation.
│ --help                               Show this message and exit.
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

Example

Nvidia T4X4 - XInference(vLLM) - Qwen1.5-7B-Chat

evalscope-perf http://172.16.33.66:9997/v1/chat/completions gpt-4-32k \
    ./datasets/open_qa.jsonl \
    --parallels 4 \
    --parallels 8 \
    --parallels 16 \
    --n 20

Ascend 910B4X8 - MindIE - DeepSeek-Coder-6.7B-Instruct

evalscope-perf http://127.0.0.1:1025/v1/chat/completions qwen \
    ./datasets/Codefuse-Evol-Instruct-Clean-data.jsonl \
    --parallels 32 \
    --parallels 64 \
    --parallels 100 \
    --parallels 128 \
    --parallels 150 \
    --parallels 200 \
    --parallels 256 \
    --parallels 300 \
    --parallels 400 \
    --parallels 500 \
    --parallels 600 \
    --parallels 700 \
    --parallels 800 \
    --parallels 900 \
    --parallels 1000 \
    --n 2000

evalscope-perf http://127.0.0.1:11434/v1/chat/completions qwen2.5:7b
./datasets/open_qa.jsonl
--parallels 1
--parallels 2
--n 4

Metadata

Release files for evalscope-perf 1.0.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for evalscope-perf 1.0.0
File Size Uploaded Provenance Actions
evalscope_perf-1.0.0.tar.gz 21.1 MB Without provenance Details for evalscope_perf-1.0.0.tar.gz

Release files / evalscope_perf-1.0.0.tar.gz

Download URL evalscope_perf-1.0.0.tar.gz
Size 21.1 MB
Tags Source
SHA-256 checksum
How to use checksums
c04cbf4f5985f5b3b0bedc22a1e82ccd7a864180cf4c9d081628d424660965b3
BLAKE2b-256 checksum
How to use checksums
804b0d944b37437aa4bc16043aff735337b914c6fb23679620d8b073f9765e27
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.10.9

Security for evalscope-perf 1.0.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page