LLM Benchmark
Project description
llm-benchmark (ollama-benchmark)
LLM Benchmark for Throughput via Ollama (Local LLMs)
Installation prerequisites
Working Ollama installation.
Installation Steps
Depending on your python setup either
pip install llm-benchmark
or
pipx install llm-benchmark
Usage for general users directly
llm_benchmark run
Installation and Usage in Video format
It's tested on Python 3.9 and above.
ollama installation with the following models installed
7B model can be run on machines with 8GB of RAM
13B model can be run on machines with 16GB of RAM
Usage explaination
On Windows, Linux, and macOS, it will detect memory RAM size to first download required LLM models.
When memory RAM size is greater than or equal to 4GB, but less than 7GB, it will check if gemma:2b exist. The program implicitly pull the model.
ollama pull deepseek-r1:1.5b
ollama pull gemma:2b
ollama pull phi:2.7b
ollama pull phi3:3.8b
When memory RAM size is greater than 7GB, but less than 15GB, it will check if these models exist. The program implicitly pull these models
ollama pull phi3:3.8b
ollama pull gemma2:9b
ollama pull mistral:7b
ollama pull llama3.1:8b
ollama pull deepseek-r1:8b
ollama pull llava:7b
When memory RAM size is greater than 15GB, but less than 31GB, it will check if these models exist. The program implicitly pull these models
ollama pull gemma2:9b
ollama pull mistral:7b
ollama pull phi4:14b
ollama pull deepseek-r1:8b
ollama pull deepseek-r1:14b
ollama pull llava:7b
ollama pull llava:13b
When memory RAM size is greater than 31GB, it will check if these models exist. The program implicitly pull these models
ollama pull phi4:14b
ollama pull deepseek-r1:14b
ollama pull deepseek-r1:32b
Python Poetry manually(advanced) installation
https://python-poetry.org/docs/#installing-manually
For developers to develop new features on Windows Powershell or on Ubuntu Linux or macOS
python3 -m venv .venv
. ./.venv/bin/activate
pip install -U pip setuptools
pip install poetry
Usage in Python virtual environment
poetry shell
poetry install
llm_benchmark hello jason
Example #1 send systeminfo and benchmark results to a remote server
llm_benchmark run
Example #2 Do not send systeminfo and benchmark results to a remote server
llm_benchmark run --no-sendinfo
Example #3 Benchmark run on explicitly given the path to the ollama executable (When you built your own developer version of ollama)
llm_benchmark run --ollamabin=~/code/ollama/ollama
Example #4 run custom benchmark models
- Create a custom benchmark file like following yaml format, replace with your own benchmark models, remember to use double quote for your model name
file_name: "custombenchmarkmodels.yml"
version: 2.0.custom
models:
- model: "deepseek-r1:1.5b"
- model: "qwen:0.5b"
- run with the flag and point to the path of custombenchmarkmodels.yml
llm_benchmark run --custombenchmark=path/to/custombenchmarkmodels.yml
Reference
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file llm_benchmark-0.4.8.tar.gz
.
File metadata
- Download URL: llm_benchmark-0.4.8.tar.gz
- Upload date:
- Size: 2.1 MB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/6.1.0 CPython/3.12.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 |
4e0b2efc7ee6af052c3f4127f34deb2948810b057cb6e14d6dc3d957f28e0cdc
|
|
MD5 |
6d668d99e74e93e75b5b7733f40674f9
|
|
BLAKE2b-256 |
a57ddefada3b74e7c15f1294ec61669f8b47c125b28e817d6fec24505216b42f
|
Provenance
The following attestation bundles were made for llm_benchmark-0.4.8.tar.gz
:
Publisher:
python-publish.yml
on aidatatools/ollama-benchmark
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1
-
Predicate type:
https://docs.pypi.org/attestations/publish/v1
-
Subject name:
llm_benchmark-0.4.8.tar.gz
-
Subject digest:
4e0b2efc7ee6af052c3f4127f34deb2948810b057cb6e14d6dc3d957f28e0cdc
- Sigstore transparency entry: 250732901
- Sigstore integration time:
-
Permalink:
aidatatools/ollama-benchmark@4f77a7be36767fc6244c108d1fbd3880fbbe5848
-
Branch / Tag:
refs/tags/v0.4.8
- Owner: https://github.com/aidatatools
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com
-
Runner Environment:
github-hosted
-
Publication workflow:
python-publish.yml@4f77a7be36767fc6244c108d1fbd3880fbbe5848
-
Trigger Event:
push
-
Statement type:
File details
Details for the file llm_benchmark-0.4.8-py3-none-any.whl
.
File metadata
- Download URL: llm_benchmark-0.4.8-py3-none-any.whl
- Upload date:
- Size: 2.1 MB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/6.1.0 CPython/3.12.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 |
9f9ec523be4f7200efee72880cfc21da6c5897a9de4f9ae18ed592be2ebf42eb
|
|
MD5 |
b30b0a3cc302d0488e1b65310c4021c0
|
|
BLAKE2b-256 |
2c046e7d41637c8e927357a2a55cdcd3fae5b9606e0435cd4a619e9c570e708c
|
Provenance
The following attestation bundles were made for llm_benchmark-0.4.8-py3-none-any.whl
:
Publisher:
python-publish.yml
on aidatatools/ollama-benchmark
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1
-
Predicate type:
https://docs.pypi.org/attestations/publish/v1
-
Subject name:
llm_benchmark-0.4.8-py3-none-any.whl
-
Subject digest:
9f9ec523be4f7200efee72880cfc21da6c5897a9de4f9ae18ed592be2ebf42eb
- Sigstore transparency entry: 250732918
- Sigstore integration time:
-
Permalink:
aidatatools/ollama-benchmark@4f77a7be36767fc6244c108d1fbd3880fbbe5848
-
Branch / Tag:
refs/tags/v0.4.8
- Owner: https://github.com/aidatatools
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com
-
Runner Environment:
github-hosted
-
Publication workflow:
python-publish.yml@4f77a7be36767fc6244c108d1fbd3880fbbe5848
-
Trigger Event:
push
-
Statement type: