Skip to main content

Python bindings for GGML

Project description

Cosmian Python bindings for ggml

PyPI version

This library provides python bindings for the examples models of ggml.

Using this library, you can deploy state-of-the-art LLMs on Intel SGX hardware with Cosmian MSE.

Available models architecture

from cosmian_py_ggml import GPTNeoX
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-neox-20b")
model = GPTNeoX("ggml_model_path")

res = model(tokenizer.encode(query))
print("Response:", tokenizer.decode(res))
from cosmian_py_ggml import Falcon
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("tiiuae/falcon-40b")
model = Falcon("ggml_model_path")

res = model(tokenizer.encode(query))
print("Response:", tokenizer.decode(res))
from cosmian_py_ggml import MPT
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("mosaicml/mpt-30b")
model = MPT("ggml_model_path")

res = model(tokenizer.encode(query))
print("Response:", tokenizer.decode(res))

Build from sources

git clone --recursive git@github.com:Cosmian/cosmian_py_ggml.git
scripts/build.sh

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

cosmian_py_ggml-1.2.0-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (330.3 kB view details)

Uploaded Python 3 manylinux: glibc 2.17+ x86-64

File details

Details for the file cosmian_py_ggml-1.2.0-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for cosmian_py_ggml-1.2.0-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 7c9b9be6a8f0493f02a0f2cd5885882e680466a49c86029a9656f4b40c9b7113
MD5 5cca2b2411c0c75f720b1d50945079f4
BLAKE2b-256 af377b040acd4e7fab7ab448b3621a849838b70f6a8335e61b562a90667487fb

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page