Python bindings for GGML
Project description
Cosmian Python bindings for ggml
This library provides python bindings for the examples models of ggml.
Using this library, you can deploy state-of-the-art LLMs on Intel SGX hardware with Cosmian MSE.
Available models architecture
from cosmian_py_ggml import GPTNeoX
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-neox-20b")
model = GPTNeoX("ggml_model_path")
res = model(tokenizer.encode(query))
print("Response:", tokenizer.decode(res))
from cosmian_py_ggml import Falcon
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("tiiuae/falcon-40b")
model = Falcon("ggml_model_path")
res = model(tokenizer.encode(query))
print("Response:", tokenizer.decode(res))
from cosmian_py_ggml import MPT
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("mosaicml/mpt-30b")
model = MPT("ggml_model_path")
res = model(tokenizer.encode(query))
print("Response:", tokenizer.decode(res))
Build from sources
git clone --recursive git@github.com:Cosmian/cosmian_py_ggml.git
scripts/build.sh
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
No source distribution files available for this release.See tutorial on generating distribution archives.
Built Distribution
File details
Details for the file cosmian_py_ggml-1.2.0-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
.
File metadata
- Download URL: cosmian_py_ggml-1.2.0-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
- Upload date:
- Size: 330.3 kB
- Tags: Python 3, manylinux: glibc 2.17+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7c9b9be6a8f0493f02a0f2cd5885882e680466a49c86029a9656f4b40c9b7113 |
|
MD5 | 5cca2b2411c0c75f720b1d50945079f4 |
|
BLAKE2b-256 | af377b040acd4e7fab7ab448b3621a849838b70f6a8335e61b562a90667487fb |