connector core built on llama.cpp
Project description
llama-core
This is a solo llama connector; being able to work independently.
install via (pip/pip3):
pip install llama-core
run it by (python/python3):
python -m llama_core
Other functions are same as llama-cpp-python; for CUDA(GPU, Nvida) and Metal(M1/M2, Apple) supported settings, please specify CMAKE_ARGS
following Abetlen's repo below; if you want to install it by source file (under releases), you should opt to do it by .tar.gz file (then build your machine-customized installable package) rather than .whl (wheel; a pre-built binary package) with an appropriate cmake tag(s).
references
build from llama-core-(version).tar.gz (examples below are for CPU)
According to the latest note inside vs code, msys64 is recommended by Microsoft; or you can opt w64devkit or etc. as source of your gcc and g++ compilers.
for windows user(s):
$env:CMAKE_GENERATOR = "MinGW Makefiles"
$env:CMAKE_ARGS = "-DCMAKE_C_COMPILER=C:/msys64/mingw64/bin/gcc.exe -DCMAKE_CXX_COMPILER=C:/msys64/mingw64/bin/g++.exe"
pip install llama-core-(version).tar.gz
In mac, xcode command line tools are recommended by Apple for dealing all coding related issue(s); or you can bypass it for your own preference.
for mac user(s):
pip3 install llama-core-(version).tar.gz
Make sure your gcc and g++ are >=11; you can check it by: gcc --version and g++ --version; if you opt to install it by wheel (.whl) file then you don't need to worry about that.)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for llama_core-0.0.4-cp312-cp312-macosx_11_0_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b61134ff3196c48ec24c521a10ae91d09dcf21ea71dde10c25fa52b2552bed96 |
|
MD5 | c8fae371b44629c8a45772dfb0ea1c7d |
|
BLAKE2b-256 | 5deea6843bed253fed2ce18cbbae670aa9491b1e551e9a947742b5c6e044d232 |
Hashes for llama_core-0.0.4-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3dc52d69aa725a7c7a92454a145f15e66df9bac84f3322a6696ddbddf92e5034 |
|
MD5 | 551cfb5f3fb6569ac13d8d1a93938fcc |
|
BLAKE2b-256 | 76a1730cdb64b7d7b7f88ff9b21d8e7378046114edaacaf601c1198461658645 |