Python binding for llama.cpp using cffi
Project description
llama-cpp-cffi
Python binding for llama.cpp using cffi
Build
#
# setup venv
#
python -m venv venv
source venv/bin/activate
pip install poetry
#
# build
#
# x86_64
poetry run cibuildwheel --output-dir wheelhouse --platform linux --arch x86_64 .
# aarch64
docker run --rm --privileged linuxkit/binfmt:v0.8
poetry run cibuildwheel --output-dir wheelhouse --platform linux --arch aarch64 .
# pyodide, pyscript, wasm (NOTE: cannot be published to PyPI)
# poetry run cibuildwheel --output-dir wheelhouse --platform pyodide .
#
# publish
#
poetry publish --dist-dir wheelhouse
#
# run demos
#
python -B examples/demo_cffi.py
python -B examples/demo_ctypes.py
python -m http.server -d examples/demo_pyonide -b "0.0.0.0" 5000
make -j llama-cli-shared llama-cli-static GGML_NO_OPENMP=1 GGML_NO_LLAMAFILE=1
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
No source distribution files available for this release.See tutorial on generating distribution archives.
Built Distributions
Close
Hashes for llama_cpp_cffi-0.0.3-cp312-cp312-musllinux_1_2_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e60af1ecd1ee89fc4d35f6313690ae62423fedd27cae7a7fe03c3c8c138a8403 |
|
MD5 | b37ba5d85ae638baae5106348ecf5afc |
|
BLAKE2b-256 | 057a91009671e481ebf24e5adab759fd7e40a60114f47011cda4c7d161239551 |
Close
Hashes for llama_cpp_cffi-0.0.3-cp312-cp312-musllinux_1_2_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5abfcae408a5664ba263a899b2750db9f71cd08ce43090151c643465a1190ac6 |
|
MD5 | 8e60ec9bea4feebb18723b6979ecfc36 |
|
BLAKE2b-256 | 26e42973928918a0080f45e4e590ae865db77e85474f0a76ef73b2cd570cc519 |
Close
Hashes for llama_cpp_cffi-0.0.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e4ce10cf3cc4fee585955cb4f0a1e5f138e30b13454ce79d19afcb2c4ed5964c |
|
MD5 | 45cd751abb33f0768a7f066c92d54460 |
|
BLAKE2b-256 | aef705af7ab51faed7e2d882f64db8ff151cedbb5f948d0ff8b2a3cfbc691fdd |
Close
Hashes for llama_cpp_cffi-0.0.3-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | baefee3535dfd8975510e448d727affd212cffb12fa47f56b5d6f48f74506359 |
|
MD5 | 442507c0f7e96d44abd3c3ba669368e9 |
|
BLAKE2b-256 | 1872cc50684033fd07f4247ad5d0df0a3f78d20e4fa243da915d4e56c60be75a |
Close
Hashes for llama_cpp_cffi-0.0.3-cp311-cp311-musllinux_1_2_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c38b61a8ad0aa429c90ba2ba5bc8d4953e953f8de24da43984bc2a60290d0571 |
|
MD5 | e1063093a857d855ba7b57355d78eaa1 |
|
BLAKE2b-256 | 63ac851e1159629fd74e4c22eb83e4cd99c07fa0b423a5b105543f810bcc9390 |
Close
Hashes for llama_cpp_cffi-0.0.3-cp311-cp311-musllinux_1_2_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 73f27416fb08d9f06418c3fafb480228d9f8ca2d3c06bcb0db79c59a27af9cff |
|
MD5 | 9e2494d278cbaec58426c2bc5eceff1b |
|
BLAKE2b-256 | eeb2eafdd3a19c6206078e185373b0632f634c9f12dfc81fa7f1f70712e3c8cd |
Close
Hashes for llama_cpp_cffi-0.0.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 927cd60c2cf6c01ba8737a3933b0e9e34412e2e3b49d9c8263445bd7e2daf3da |
|
MD5 | 30c3aa0fa264c5a9de4042739ca6f170 |
|
BLAKE2b-256 | 8d50845d92ebc3f84e6dcb176c02a8abe28c79e89f7af547448f8f78d94981fd |
Close
Hashes for llama_cpp_cffi-0.0.3-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c26901db3b09ad395044edcf6f9d928e182783e0a312a8ea1bbcc7d3abbda306 |
|
MD5 | d6f53c41fec5506064719be68056ed6c |
|
BLAKE2b-256 | 5fb9ae713cace490bf9e0df9170b102f0bd9e5366bbbcfea2194002efc5be657 |
Close
Hashes for llama_cpp_cffi-0.0.3-cp310-cp310-musllinux_1_2_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c972286964683cd03df5e836ebd915a31461f7ddeb6fe2ae53f80a0e761f9aee |
|
MD5 | 420e6a45b08253b48a5eaefb8ce48f40 |
|
BLAKE2b-256 | 5af5a118b8fa3ade5903f4a60ea14f89626badf9819ff694b42ff19612e6c0f2 |
Close
Hashes for llama_cpp_cffi-0.0.3-cp310-cp310-musllinux_1_2_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 45a0844a465abc61b1c64004c86fb5ace11024a67fbf91c4880c369077b53deb |
|
MD5 | 0618f2bde65df823736a54398363b5e6 |
|
BLAKE2b-256 | 1ae9430ef629bab8cbce151f5bc5801123fbc330b409fc0107eb1446bf202cd2 |
Close
Hashes for llama_cpp_cffi-0.0.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ebdf00863f7c9c10eb7d23d3c8f420c39599e163aa448a96205e5595223b1fd2 |
|
MD5 | bc0b90ff911ca0b6c3a3f17943911af6 |
|
BLAKE2b-256 | 7acc42e43c13bd48ac62f3a91102458fa173c83ae7f75189948b84ed36d6f271 |
Close
Hashes for llama_cpp_cffi-0.0.3-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | de2ae761ec654006744530cd20cf40a61206bf78f96d874c822f2da72bd4f08f |
|
MD5 | bf13a88bccb5d440058c690965f92768 |
|
BLAKE2b-256 | b50232ce97f5aca4cd30c9a887f60286a419ce84a2784f1bb9f84895576b3b44 |