Python bindings for ggml
Project description
Python bindings for ggml
Python bindings for the ggml
tensor library for machine learning.
⚠️ This project is in a very early state and currently only offers the basic low-level bindings to ggml
Documentation
Installation
Requirements
- Python 3.7+
- C compiler (gcc, clang, msvc, etc)
You can install ggml-python
using pip
:
pip install ggml-python
This will compile ggml using cmake which requires a c compiler installed on your system.
To build ggml with specific features (ie. OpenBLAS, GPU Support, etc) you can pass specific flags through the CMAKE_ARGS
environment variable. For example to install ggml-python with cuBLAS support you can run:
CMAKE_ARGS="-DGGML_CUBLAS=ON" pip install ggml-python
Options
Option | Description | Default |
---|---|---|
GGML_CUBLAS |
Enable cuBLAS support | OFF |
GGML_CLBLAST |
Enable CLBlast support | OFF |
GGML_OPENBLAS |
Enable OpenBLAS support | OFF |
GGML_METAL |
Enable Metal support | OFF |
Usage
import ggml
import ctypes
# Allocate a new context with 16 MB of memory
params = ggml.ggml_init_params(mem_size=16 * 1024 * 1024, mem_buffer=None)
ctx = ggml.ggml_init(params=params)
# Instantiate tensors
x = ggml.ggml_new_tensor_1d(ctx, ggml.GGML_TYPE_F32, 1)
a = ggml.ggml_new_tensor_1d(ctx, ggml.GGML_TYPE_F32, 1)
b = ggml.ggml_new_tensor_1d(ctx, ggml.GGML_TYPE_F32, 1)
# Use ggml operations to build a computational graph
x2 = ggml.ggml_mul(ctx, x, x)
f = ggml.ggml_add(ctx, ggml.ggml_mul(ctx, a, x2), b)
gf = ggml.ggml_new_graph(ctx)
ggml.ggml_build_forward_expand(gf, f)
# Set the input values
ggml.ggml_set_f32(x, 2.0)
ggml.ggml_set_f32(a, 3.0)
ggml.ggml_set_f32(b, 4.0)
# Compute the graph
ggml.ggml_graph_compute_with_ctx(ctx, gf, 1)
# Get the output value
output = ggml.ggml_get_f32_1d(f, 0)
assert output == 16.0
# Free the context
ggml.ggml_free(ctx)
Troubleshooting
If you are having trouble installing ggml-python
or activating specific features please try to install it with the --verbose
and --no-cache-dir
flags to get more information about any issues:
[options] pip install ggml-python --verbose --no-cache-dir --force-reinstall --upgrade
License
This project is licensed under the terms of the MIT license.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.