tmesh 0.1.17

TensorMesh CLI tool suite

Documentation

docs.tmesh.ai

Installation

pip install tmesh

CLI Tree

1. tmesh-cli benchmark

Format: tmesh-cli benchmark --endpoint "<YOUR_OPEN_AI_API_ENDPOINT>" --api-key "<OPTIONAL_API_KEY>" [--model "<OPTIONAL_MODEL>"]

Example:

tmesh-cli benchmark --endpoint "http://89.169.111.29:30080/" --api-key "vllm_sk_555a1b7ff3e0f617b1240375ea411c2a5f08d2666fcdc718075f66c9"

The model at the endpoint will automatically be discovered (or you can manually pass it in) and the benchmark will run ad infinitum.

Discovery and Configuration:

endpoint: http://localhost:8000/v1/
found model: Qwen/Qwen3-30B-A3B-Instruct-2507
offload_size: 100
Workload Specifications:
Model: Qwen/Qwen3-30B-A3B-Instruct-2507
Number of Contexts: 30
Number of Questions per Context: 30
Max Inflight Requests (Load-Balancing): 10
Input Length: 32000
Output Length: 100

Continually send M long contexts that will always have N randomly generated questions appended to them.

Hardcoded Configurations:

• 32k token context length
• 100 token output length
• TensorMesh SaaS configurations in: src/benchmark/model_configs.json

Discovered Configurations:

• Model name (must be one of the ones supported by LMIgnite inside of model_configs.json)
• Number of contexts
• Number of questions per context
• Maximum Inflight Requests

Refreshing Logs every 5 seconds

Elapsed Time: 165.0652596950531
Total Number of Requests Processed: 280
QPS: 1.696298788232491
Global Average TTFT: 3.77425986954144
Global Average ITL: 0.01485872107813842
Global Average Prefill Throughput: 22871.74402415523
Global Average Decode Throughput: 104.11856198232987
Requests Processed in Interval: 0
Interval Average TTFT: 2.5995567440986633
Interval Average ITL: 0.012132209007956864
Interval Average Prefill Throughput: 25740.820827727515
Interval Average Decode Throughput: 90.82104365217528

2.