llama-cpp-bin
Pre-built llama.cpp server binaries as a py package. Install a wheel for your platform and run it.
Install
Pre-built wheels (recommended)
pip install --index-url https://vladlearns.github.io/llama-cpp-bin/whl/cpu llama-cpp-bin
pip install --index-url https://vladlearns.github.io/llama-cpp-bin/whl/cu124 llama-cpp-bin
pip install --index-url https://vladlearns.github.io/llama-cpp-bin/whl/cu131 llama-cpp-bin
pip install --index-url https://vladlearns.github.io/llama-cpp-bin/whl/rocm llama-cpp-bin
pip install --index-url https://vladlearns.github.io/llama-cpp-bin/whl/vulkan llama-cpp-bin
Pin to a specific version:
pip install --index-url https://vladlearns.github.io/llama-cpp-bin/whl/cu124 llama-cpp-bin==9095.0.0
PyPI (builds from source)
If no pre-built wheel matches your platform, pip falls back to building from the sdist on PyPI:
pip install llama-cpp-bin
You will need CMake, a c++ compiler, and the llama.cpp source submodule.
Dev
git clone --recurse-submodules https://github.com/vladlearns/llama-cpp-bin
cd llama-cpp-bin
CMAKE_ARGS="-DGGML_CUDA=ON" pip install -v .
Run
CLI:
llama-cpp-server -m your-model.gguf --port 8080
Python:
from llama_cpp_bin import run_server
proc = run_server("your-model.gguf", port=8080)
proc.wait()
Or get the binary path and run it yourself:
import llama_cpp_bin
import subprocess
binary = llama_cpp_bin.get_binary_path()
subprocess.Popen([binary, "--model", "your-model.gguf"])
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
llama_cpp_bin-10227.0.0.tar.gz
(36.0 MB
view details)
File details
Details for the file llama_cpp_bin-10227.0.0.tar.gz.
File metadata
- Download URL: llama_cpp_bin-10227.0.0.tar.gz
- Upload date:
- Size: 36.0 MB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
a98b0151c08ef8dd1ca4a9f9913acf79b23da0566a885e9de0374519f5c7e563
|
|
| MD5 |
b8aa896e3e5bbf0082f60ebe4d4b42e1
|
|
| BLAKE2b-256 |
11b0ed4b6c7082f609c62e0c037242f826d777e5c418ed9559bfbf95c0f255da
|
Provenance
The following attestation bundles were made for llama_cpp_bin-10227.0.0.tar.gz:
Publisher:
build-everything.yml on vladlearns/llama-cpp-bin
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
llama_cpp_bin-10227.0.0.tar.gz -
Subject digest:
a98b0151c08ef8dd1ca4a9f9913acf79b23da0566a885e9de0374519f5c7e563 - Sigstore transparency entry: 2324106717
- Sigstore integration time:
-
Permalink:
vladlearns/llama-cpp-bin@03d9e8860992507eb4de9e2254a3103cfc889034 -
Branch / Tag:
refs/heads/main - Owner: https://github.com/vladlearns
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
build-everything.yml@03d9e8860992507eb4de9e2254a3103cfc889034 -
Trigger Event:
workflow_dispatch
-
Statement type:
