DuneBench
dunebench is a lightweight, local benchmarking tool for GGUF models. It allows you to evaluate Large Language Models (LLMs) across a variety of domains—including logic, coding, math, and common sense—using llama-cpp-python.
Installation
Install dunebench with pip
pip install dunebench
or install EXE with this link
Features
- Local Evaluation: Runs entirely on your machine using GGUF models.
- GPU Accelerated: Offload layers to your GPU for faster testing.
- Multi-Domain Support: Includes 8 distinct benchmarks (Math, Coding, Science, etc.).
Usage/Examples
python -m dunebench --model "path/to/model.gguf" --task science --limit 20
Arguments
| Argument | Description | Default |
|---|---|---|
--model |
Path to your .gguf model file |
Required |
--task |
The benchmark task to run | Required |
--limit |
Number of samples to test | 10 |
Tasks
| Task Name | Dataset Used | Domain | Type |
|---|---|---|---|
| science | ai2_arc (Challenge) | Scientific Reasoning | Multiple Choice |
| math | gsm8k | Math Word Problems | Generation |
| programming | mbpp (Sanitized) | Python Coding | Code Generation |
| physical_logic | piqa | Physical Commonsense | Multiple Choice |
| common_sense | openbookqa | General Knowledge | Multiple Choice |
| logic | winogrande | Ambiguity Resolution | Multiple Choice |
| grammar | glue (CoLA) | Linguistic Acceptability | Multiple Choice |
| nlp | hellaswag | Sentence Completion | Multiple Choice |
License
Authors
1 maintainer
Metadata
1 maintainer
Release files for dunebench 0.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| dunebench-0.4.tar.gz | 6.5 kB | Without provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| dunebench-0.4-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 13.2 kB
Release files / dunebench-0.4.tar.gz
| Download URL | dunebench-0.4.tar.gz |
|---|---|
| Size | 6.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
fc2321abb0616a6234a353b635829801ba62b8c736236bbfa49733e573bfdc3b
|
|
BLAKE2b-256 checksum How to use checksums |
9fc6f68d9e7574c81120c89ba02c742a92fec1019c9492a27d6c799cb0d24ffb
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.10.11
|
Release files / dunebench-0.4-py3-none-any.whl
| Download URL | dunebench-0.4-py3-none-any.whl |
|---|---|
| Size | 6.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
2825a82285b70dfcf9712a1210a9bb17b8b03729050eaa33f3bbfe61dbaee9a8
|
|
BLAKE2b-256 checksum How to use checksums |
889c8549d067ec6f51f1e67003c499c2f3d8363e782a82afd707da62981db2c1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.10.11
|
Security for dunebench 0.4
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
