Skip to main content

Python bindings for GGML

Project description

Cosmian Python bindings for ggml

PyPI version

This library provides python bindings for the examples models of ggml.

Available models architecture

from cosmian_py_ggml import gpt_neox
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-neox-20b")
model = gpt_neox.load_model("ggml_model_path")

res = gpt_neox.generate(model, tokenizer.encode(query))
print("Response:", tokenizer.decode(res))

gpt_neox.free_model(model)
from cosmian_py_ggml import falcon
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("tiiuae/falcon-7b")
model = falcon.load_model("ggml_model_path")

res = falcon.generate(model, tokenizer.encode(query))
print("Response:", tokenizer.decode(res))

falcon.free_model(model)

Build from sources

git clone --recursive git@github.com:Cosmian/cosmian_py_ggml.git
scripts/build.sh

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

cosmian_py_ggml-0.5.0-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (315.7 kB view hashes)

Uploaded Python 3 manylinux: glibc 2.17+ x86-64

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page