Racing Llama Benchmark is a benchmark tool for comparing llama.cpp performance against different versions and hardware.

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for soleblaze from gravatar.com soleblaze

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: MIT License (MIT)
  • Author: Soleblaze
  • Requires: Python >=3.8.1, <4.0.0
Classifiers

Project description

rlb

The Racing Llama Benchmark (rlb) is designed to provide consistent LLM benchmarking across Linux, MacOS, and Windows.

NOTE: This project is in Alpha state and is currently being developed for MacOS. Linux and Windows support will be added next.

TODO

  • Support Running llama.cpp against multiple parameter and quantization types
  • Graph output
  • Comparing thread counts
  • Improved benchmark prompts
  • Memory benchmarking
  • CPU benchmarking
  • Possible HPL-MxP support
  • Possible jitter measurements
  • JSON output
  • Versions in output, such as llama.cpp, python-llama-cpp
  • Linux support
  • Windows Support

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for soleblaze from gravatar.com soleblaze

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: MIT License (MIT)
  • Author: Soleblaze
  • Requires: Python >=3.8.1, <4.0.0
Classifiers

Release history Release notifications | RSS feed

This version

0.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

rlb-0.0.1.tar.gz (4.5 kB view details)

Uploaded Source

Built Distribution

rlb-0.0.1-py3-none-any.whl (5.6 kB view details)

Uploaded Python 3

File details

Details for the file rlb-0.0.1.tar.gz.

File metadata

  • Download URL: rlb-0.0.1.tar.gz
  • Upload date:
  • Size: 4.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.5.1 CPython/3.11.4 Darwin/22.5.0

File hashes

Hashes for rlb-0.0.1.tar.gz
Algorithm Hash digest
SHA256 a8caae26ad6cfc4cc330e625f44edb39ffe8631b0d93a0e46a2752dafe391d86
MD5 6ebcc161afe2763a855f7092e472df59
BLAKE2b-256 2700cd1cc58f487382b9e590e63da92620742bf90ae938cf947a0a350b64781d

See more details on using hashes here.

File details

Details for the file rlb-0.0.1-py3-none-any.whl.

File metadata

  • Download URL: rlb-0.0.1-py3-none-any.whl
  • Upload date:
  • Size: 5.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.5.1 CPython/3.11.4 Darwin/22.5.0

File hashes

Hashes for rlb-0.0.1-py3-none-any.whl
Algorithm Hash digest
SHA256 b7b71fba8cc8184caf4deae720d0ba0832819f6509c028170dd94b60e27a0d00
MD5 66bb41fc1183336b74309b6c348076b2
BLAKE2b-256 490b14aad5ad3f267826acb976ec04a7883e6c85fbe5fabc3653cf2363529e06

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page