Skip to main content

LLM Benchmark

Project description

llm-benchmark (ollama-benchmark)

LLM Benchmark for Throughput via Ollama (Local LLMs)

Installation prerequisites

Working Ollama installation.

Installation Steps

Depending on your python setup either

pip install llm-benchmark

or

pipx install llm-benchmark

Usage for general users directly

llm_benchmark run

Installation and Usage in Video format

llm-benchmark

It's tested on Python 3.9 and above.

ollama installation with the following models installed

7B model can be run on machines with 8GB of RAM

13B model can be run on machines with 16GB of RAM

Usage explaination

On Windows, Linux, and macOS, it will detect memory RAM size to first download required LLM models.

When memory RAM size is greater than or equal to 4GB, but less than 7GB, it will check if gemma:2b exist. The program implicitly pull the model.

ollama pull deepseek-r1:1.5b
ollama pull gemma:2b
ollama pull phi:2.7b
ollama pull phi3:3.8b

When memory RAM size is greater than 7GB, but less than 15GB, it will check if these models exist. The program implicitly pull these models

ollama pull phi3:3.8b
ollama pull gemma2:9b
ollama pull mistral:7b
ollama pull llama3.1:8b
ollama pull deepseek-r1:8b
ollama pull llava:7b

When memory RAM size is greater than 15GB, but less than 31GB, it will check if these models exist. The program implicitly pull these models

ollama pull gemma2:9b
ollama pull mistral:7b
ollama pull phi4:14b
ollama pull deepseek-r1:8b
ollama pull deepseek-r1:14b
ollama pull llava:7b
ollama pull llava:13b

When memory RAM size is greater than 31GB, it will check if these models exist. The program implicitly pull these models

ollama pull phi4:14b
ollama pull deepseek-r1:14b
ollama pull deepseek-r1:32b

Python Poetry manually(advanced) installation

https://python-poetry.org/docs/#installing-manually

For developers to develop new features on Windows Powershell or on Ubuntu Linux or macOS

python3 -m venv .venv
. ./.venv/bin/activate
pip install -U pip setuptools
pip install poetry

Usage in Python virtual environment

poetry shell
poetry install
llm_benchmark hello jason

Example #1 send systeminfo and benchmark results to a remote server

llm_benchmark run

Example #2 Do not send systeminfo and benchmark results to a remote server

llm_benchmark run --no-sendinfo

Example #3 Benchmark run on explicitly given the path to the ollama executable (When you built your own developer version of ollama)

llm_benchmark run --ollamabin=~/code/ollama/ollama

Example #4 run custom benchmark models

  1. Create a custom benchmark file like following yaml format, replace with your own benchmark models, remember to use double quote for your model name
file_name: "custombenchmarkmodels.yml"
version: 2.0.custom
models:
  - model: "deepseek-r1:1.5b"
  - model: "qwen:0.5b"
  1. run with the flag and point to the path of custombenchmarkmodels.yml
llm_benchmark run --custombenchmark=path/to/custombenchmarkmodels.yml

Reference

Ollama

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

llm_benchmark-0.4.8.tar.gz (2.1 MB view details)

Uploaded Source

Built Distribution

llm_benchmark-0.4.8-py3-none-any.whl (2.1 MB view details)

Uploaded Python 3

File details

Details for the file llm_benchmark-0.4.8.tar.gz.

File metadata

  • Download URL: llm_benchmark-0.4.8.tar.gz
  • Upload date:
  • Size: 2.1 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for llm_benchmark-0.4.8.tar.gz
Algorithm Hash digest
SHA256 4e0b2efc7ee6af052c3f4127f34deb2948810b057cb6e14d6dc3d957f28e0cdc
MD5 6d668d99e74e93e75b5b7733f40674f9
BLAKE2b-256 a57ddefada3b74e7c15f1294ec61669f8b47c125b28e817d6fec24505216b42f

See more details on using hashes here.

Provenance

The following attestation bundles were made for llm_benchmark-0.4.8.tar.gz:

Publisher: python-publish.yml on aidatatools/ollama-benchmark

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file llm_benchmark-0.4.8-py3-none-any.whl.

File metadata

  • Download URL: llm_benchmark-0.4.8-py3-none-any.whl
  • Upload date:
  • Size: 2.1 MB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for llm_benchmark-0.4.8-py3-none-any.whl
Algorithm Hash digest
SHA256 9f9ec523be4f7200efee72880cfc21da6c5897a9de4f9ae18ed592be2ebf42eb
MD5 b30b0a3cc302d0488e1b65310c4021c0
BLAKE2b-256 2c046e7d41637c8e927357a2a55cdcd3fae5b9606e0435cd4a619e9c570e708c

See more details on using hashes here.

Provenance

The following attestation bundles were made for llm_benchmark-0.4.8-py3-none-any.whl:

Publisher: python-publish.yml on aidatatools/ollama-benchmark

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page