Skip to main content

Python bindings for GPT4All

Project description

Python GPT4All

This package contains a set of Python bindings around the llmodel C-API.

Package on PyPI: https://pypi.org/project/gpt4all/

Documentation

https://docs.gpt4all.io/gpt4all_python.html

Installation

The easiest way to install the Python bindings for GPT4All is to use pip:

pip install gpt4all

This will download the latest version of the gpt4all package from PyPI.

Local Build

As an alternative to downloading via pip, you may build the Python bindings from source.

Prerequisites

You will need a compiler. On Windows, you should install Visual Studio with the C++ Development components. On macOS, you will need the full version of Xcode—Xcode Command Line Tools lacks certain required tools. On Linux, you will need a GCC or Clang toolchain with C++ support.

On Windows and Linux, building GPT4All with full GPU support requires the Vulkan SDK and the latest CUDA Toolkit.

Building the python bindings

  1. Clone GPT4All and change directory:
git clone --recurse-submodules https://github.com/nomic-ai/gpt4all.git
cd gpt4all/gpt4all-backend
  1. Build the backend.

If you are using Windows and have Visual Studio installed:

cmake -B build
cmake --build build --parallel --config RelWithDebInfo

For all other platforms:

cmake -B build -DCMAKE_BUILD_TYPE=RelWithDebInfo
cmake --build build --parallel

RelWithDebInfo is a good default, but you can also use Release or Debug depending on the situation.

  1. Install the Python package:
cd ../gpt4all-bindings/python
pip install -e .

Usage

Test it out! In a Python script or console:

from gpt4all import GPT4All
model = GPT4All("orca-mini-3b-gguf2-q4_0.gguf")
output = model.generate("The capital of France is ", max_tokens=3)
print(output)

GPU Usage

from gpt4all import GPT4All
model = GPT4All("orca-mini-3b-gguf2-q4_0.gguf", device='gpu') # device='amd', device='intel'
output = model.generate("The capital of France is ", max_tokens=3)
print(output)

Troubleshooting a Local Build

  • If you're on Windows and have compiled with a MinGW toolchain, you might run into an error like:

    FileNotFoundError: Could not find module '<...>\gpt4all-bindings\python\gpt4all\llmodel_DO_NOT_MODIFY\build\libllmodel.dll'
    (or one of its dependencies). Try using the full path with constructor syntax.
    

    The key phrase in this case is "or one of its dependencies". The Python interpreter you're using probably doesn't see the MinGW runtime dependencies. At the moment, the following three are required: libgcc_s_seh-1.dll, libstdc++-6.dll and libwinpthread-1.dll. You should copy them from MinGW into a folder where Python will see them, preferably next to libllmodel.dll.

  • Note regarding the Microsoft toolchain: Compiling with MSVC is possible, but not the official way to go about it at the moment. MSVC doesn't produce DLLs with a lib prefix, which the bindings expect. You'd have to amend that yourself.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

gpt4all-2.8.2-py3-none-win_amd64.whl (119.6 MB view details)

Uploaded Python 3 Windows x86-64

gpt4all-2.8.2-py3-none-manylinux1_x86_64.whl (121.6 MB view details)

Uploaded Python 3

gpt4all-2.8.2-py3-none-macosx_10_15_universal2.whl (6.6 MB view details)

Uploaded Python 3 macOS 10.15+ universal2 (ARM64, x86-64)

File details

Details for the file gpt4all-2.8.2-py3-none-win_amd64.whl.

File metadata

  • Download URL: gpt4all-2.8.2-py3-none-win_amd64.whl
  • Upload date:
  • Size: 119.6 MB
  • Tags: Python 3, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.12.3

File hashes

Hashes for gpt4all-2.8.2-py3-none-win_amd64.whl
Algorithm Hash digest
SHA256 a164674943df732808266e5bf63332fadef95eac802c201b47c7b378e5bd9f45
MD5 ab7df157584bcbd49fe6d3a749392e36
BLAKE2b-256 0492dd9dd077f0fc61218e4116a7154958eff36f06893506ad3290802218b1c9

See more details on using hashes here.

File details

Details for the file gpt4all-2.8.2-py3-none-manylinux1_x86_64.whl.

File metadata

File hashes

Hashes for gpt4all-2.8.2-py3-none-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 c61e977afa72475c076211fd9b59a88334fc0062615be8c2f7716363a21fbafa
MD5 e44b87956d6283fda8ebd128082d55fd
BLAKE2b-256 9360ef1efd83c5757a69e8ed842b8a12b091e94a9bb2ceacc8b6e1c2e6487a77

See more details on using hashes here.

File details

Details for the file gpt4all-2.8.2-py3-none-macosx_10_15_universal2.whl.

File metadata

File hashes

Hashes for gpt4all-2.8.2-py3-none-macosx_10_15_universal2.whl
Algorithm Hash digest
SHA256 41d6de60fe1f9c9a29bcbb585c5eb39083893cf00db0b836de1f2d88e2fbbb17
MD5 a798ac37c060185a670b519a96762720
BLAKE2b-256 1992bad46a9bc5b993727045d09ce35e2dcc3939be02f47acfc99b2be0d33b33

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page