gguf connector core built on llama.cpp
Project description
llama-core
This is a solo llama connector also; being able to work independently.
install via (pip/pip3):
pip install llama-core
run it by (python/python3):
python -m llama_core
Prompt to user interface selection menu above; while chosen, GGUF file(s) in the current directory will be searched and detected (if any) as below.
include interface selector to your code by adding:
from llama_core import menu
include gguf reader to your code by adding:
from llama_core import reader
include gguf writer to your code by adding:
from llama_core import writer
remark(s)
Other functions are same as llama-cpp-python; for CUDA(GPU, Nvida) and Metal(M1/M2, Apple) supported settings, please specify CMAKE_ARGS
following Abetlen's repo below; if you want to install it by source file (under releases), you should opt to do it by .tar.gz file (then build your machine-customized installable package) rather than .whl (wheel; a pre-built binary package) with an appropriate cmake tag(s).
references
repo llama-cpp-python llama.cpp page gguf.us
build from llama_core-(version).tar.gz (examples below are for CPU)
According to the latest note inside vs code, msys64 was recommended by Microsoft; or you could opt w64devkit or etc. as source/location of your gcc and g++ compilers.
for windows user(s):
$env:CMAKE_GENERATOR = "MinGW Makefiles"
$env:CMAKE_ARGS = "-DCMAKE_C_COMPILER=C:/msys64/mingw64/bin/gcc.exe -DCMAKE_CXX_COMPILER=C:/msys64/mingw64/bin/g++.exe"
pip install llama_core-(version).tar.gz
In mac, xcode command line tools were recommended by Apple for dealing all coding related issue(s); or you could bypass it for your own good/preference.
for mac user(s):
pip3 install llama_core-(version).tar.gz
Make sure your gcc and g++ are >=11; you can check it by: gcc --version and g++ --version; other setting(s) include: cmake>=3.21, etc.; however, if you opt to install it by the pre-built wheel (.whl) file then you don't need to worry about that.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for llama_core-0.2.8-cp312-cp312-macosx_11_0_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3a77ae7d170e69b97f1c50273e6eaa26f321e230bf42239bb3ceed64f330a07b |
|
MD5 | 0bea137d8d99f4a01b45fd90011b4aef |
|
BLAKE2b-256 | 67542096a4572df8a70be66fd6d989368b262553ce478186323eacff14db7e97 |
Hashes for llama_core-0.2.8-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5cca91761c78bde5926c45f4cd84d79f0001f4d8be0c63b635f6b25350a7c29c |
|
MD5 | 1552040c4bdc77c5b58dafc5c72221b4 |
|
BLAKE2b-256 | 2d0e59bc544228973f4dce50b5bf39399be73c8589a026b03a5620d22f4e2ee2 |