veeksha 0.2.2

A framework for holistic evaluation of LLM Inference Systems

Veeksha

Veeksha is a high-fidelity benchmarking framework for LLM inference systems. Whether you're optimizing a production deployment, comparing serving backends, or running capacity planning experiments, Veeksha lets you measure what matters to you: realistic multi-turn conversations, agentic workflows, high-frequency stress tests, or targeted microbenchmarks. One tool, any workload.

From isolated requests to complex agentic sessions, Veeksha captures the full complexity of modern LLM workloads.

👉 Why Veeksha? — Learn what sets Veeksha apart
📚 Documentation — Full guides and API reference

Quick start

In a fresh environment (Python 3.14t recommended for true parallelism):

Install from PyPI:

pip install veeksha

Run a benchmark against an OpenAI-compatible endpoint:

python -Xgil=0 -m veeksha.benchmark \
    --client-type openai_chat_completions \
    --openai-chat-completions-client-api-base http://localhost:8000/v1 \
    --openai-chat-completions-client-model meta-llama/Llama-3.2-1B-Instruct \
    --traffic-scheduler-type rate \
    --rate-traffic-scheduler-interval-generator-type poisson \
    --rate-traffic-scheduler-poisson-interval-generator-arrival-rate 5.0 \
    --runtime-benchmark-timeout 60

Or use a YAML configuration file:

python -Xgil=0 -m veeksha.benchmark --benchmark-config-from-file my_benchmark.veeksha.yml

Installation from source

git clone https://github.com/project-vajra/veeksha.git
cd veeksha

# Install uv if needed
curl -LsSf https://astral.sh/uv/install.sh | sh

# Create environment (Python 3.14t recommended for true parallelism)
uv venv --python 3.14t
source .venv/bin/activate
uv pip install -e .