llm-benchmark (ollama-benchmark)
LLM Benchmark for Throughput via Ollama (Local LLMs)
Installation Steps
pip install llm-bench
Usage for general users directly
llm_bench run
ollama installation with the following models installed
7B model can be run on machines with 8GB of RAM
13B model can be run on machines with 16GB of RAM
Usage explaination
On Windows, Linux, and macOS, it will detect memory RAM size to first download required LLM models.
When memory RAM size is greater than or equal to 4GB, but less than 7GB, it will check if gemma:2b exist. The program implicitly pull the model.
ollama pull gemma:2b
When memory RAM size is greater than 7GB, but less than 15GB, it will check if these models exist. The program implicitly pull these models
ollama pull gemma:2b
ollama pull gemma:7b
ollama pull mistral:7b
ollama pull llama2:7b
ollama pull llava:7b
When memory RAM siz is greater than 15GB, it will check if these models exist. The program implicitly pull these models
ollama pull gemma:2b
ollama pull gemma:7b
ollama pull mistral:7b
ollama pull llama2:7b
ollama pull llama2:13b
ollama pull llava:7b
ollama pull llava:13b
Python Poetry manually(advanced) installation
https://python-poetry.org/docs/#installing-manually
For developers to develop new features on Windows Powershell or on Ubuntu Linux or macOS
python3 -m venv .venv
. ./.venv/bin/activate
pip install -U pip setuptools
pip install poetry
Usage in Python virtual environment
poetry shell
poetry install
llm_benchmark hello jason
Example #1 send systeminfo and benchmark results to a remote server
llm_bench run
Example #2 Do not send systeminfo and benchmark results to a remote server
llm_bench run --no-sendinfo
Example #3 Benchmark run on explicitly given the path to the ollama executable (When you built your own developer version of ollama)
llm_bench run --ollamabin=~/code/ollama/ollama
Reference
1 maintainer
Metadata
1 maintainer
Release files for llm-bench 0.4.32
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| llm_bench-0.4.32.tar.gz | 14.0 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| llm_bench-0.4.32-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 33.9 kB
Release files / llm_bench-0.4.32.tar.gz
| Download URL | llm_bench-0.4.32.tar.gz |
|---|---|
| Size | 14.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
e2ba3787f872dd96bf29fe9198b5e8981af2f3db6de3ceac05b4d1dc4e026a4b
|
|
BLAKE2b-256 checksum How to use checksums |
0b9324e32e3430f7d397862edbabb00a6fe09313b865dcc34aaa1cf433f3d7e2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.1.1 CPython/3.10.8
|
Release files / llm_bench-0.4.32-py3-none-any.whl
| Download URL | llm_bench-0.4.32-py3-none-any.whl |
|---|---|
| Size | 19.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
f76d9fe5687ad72a7c5ad714e69a9f6e14bfbba11ba528ade9d49d044c340dd6
|
|
BLAKE2b-256 checksum How to use checksums |
6ee3ef108ae54c3e946f4aa84c08df42b00252212dc0eccd78c3944f0609904f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.1.1 CPython/3.10.8
|
