Python GPT4All
This package contains a set of Python bindings around the llmodel C-API.
Package on PyPI: https://pypi.org/project/gpt4all/
Documentation
https://docs.gpt4all.io/gpt4all_python.html
Installation
The easiest way to install the Python bindings for GPT4All is to use pip:
pip install gpt4all
This will download the latest version of the gpt4all package from PyPI.
Local Build
As an alternative to downloading via pip, you may build the Python bindings from source.
Prerequisites
On Windows and Linux, building GPT4All requires the complete Vulkan SDK. You may download it from here: https://vulkan.lunarg.com/sdk/home
macOS users do not need Vulkan, as GPT4All will use Metal instead.
Building the python bindings
- Clone GPT4All and change directory:
git clone --recurse-submodules https://github.com/nomic-ai/gpt4all.git
cd gpt4all/gpt4all-backend
- Build the backend.
If you are using Windows and have Visual Studio installed:
cmake -B build
cmake --build build --parallel --config RelWithDebInfo
For all other platforms:
cmake -B build -DCMAKE_BUILD_TYPE=RelWithDebInfo
cmake --build build --parallel
RelWithDebInfo is a good default, but you can also use Release or Debug depending on the situation.
- Install the Python package:
cd ../../gpt4all-bindings/python
pip install -e .
Usage
Test it out! In a Python script or console:
from gpt4all import GPT4All
model = GPT4All("orca-mini-3b-gguf2-q4_0.gguf")
output = model.generate("The capital of France is ", max_tokens=3)
print(output)
GPU Usage
from gpt4all import GPT4All
model = GPT4All("orca-mini-3b-gguf2-q4_0.gguf", device='gpu') # device='amd', device='intel'
output = model.generate("The capital of France is ", max_tokens=3)
print(output)
Troubleshooting a Local Build
-
If you're on Windows and have compiled with a MinGW toolchain, you might run into an error like:
FileNotFoundError: Could not find module '<...>\gpt4all-bindings\python\gpt4all\llmodel_DO_NOT_MODIFY\build\libllmodel.dll' (or one of its dependencies). Try using the full path with constructor syntax.The key phrase in this case is "or one of its dependencies". The Python interpreter you're using probably doesn't see the MinGW runtime dependencies. At the moment, the following three are required:
libgcc_s_seh-1.dll,libstdc++-6.dllandlibwinpthread-1.dll. You should copy them from MinGW into a folder where Python will see them, preferably next tolibllmodel.dll. -
Note regarding the Microsoft toolchain: Compiling with MSVC is possible, but not the official way to go about it at the moment. MSVC doesn't produce DLLs with a
libprefix, which the bindings expect. You'd have to amend that yourself.
2 maintainers
Release files for gpt4all 2.5.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distributions (wheels)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| gpt4all-2.5.1-py3-none-win_amd64.whl | Python 3 | none | Windows x86-64 | Details |
| gpt4all-2.5.1-py3-none-manylinux1_x86_64.whl | Python 3 | none | Linux glibc 2.5+ x86-64 | Details |
| gpt4all-2.5.1-py3-none-macosx_10_15_universal2.whl | Python 3 | none | macOS 10.15+ universal2 (ARM64, x86-64) | Details |
Total release size: 16.0 MB
Release files / gpt4all-2.5.1-py3-none-win_amd64.whl
| Download URL | gpt4all-2.5.1-py3-none-win_amd64.whl |
|---|---|
| Size | 6.3 MB |
| Tags | Python 3 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
28b3d2dbc0dcdb731ade227a098cbba8201f98bb9871811fa656811e93302737
|
|
BLAKE2b-256 checksum How to use checksums |
1536a227f0ce3d2bf1cc516e017311bdda89321e9536c75b692e1272cc885121
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.0.0 CPython/3.8.12
|
Release files / gpt4all-2.5.1-py3-none-manylinux1_x86_64.whl
| Download URL | gpt4all-2.5.1-py3-none-manylinux1_x86_64.whl |
|---|---|
| Size | 3.9 MB |
| Tags | Linux glibc 2.5+ x86-64 Python 3 |
|
SHA-256 checksum How to use checksums |
50c41cf0b5cd1c5701d67075f002aa924abd3a99cf6f9f41864c78f65abb8cd0
|
|
BLAKE2b-256 checksum How to use checksums |
af6f1f033f7d0b6711d19abc928b193c3db301fe2b7945dc2da5e683bce789f4
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.0.0 CPython/3.8.12
|
Release files / gpt4all-2.5.1-py3-none-macosx_10_15_universal2.whl
| Download URL | gpt4all-2.5.1-py3-none-macosx_10_15_universal2.whl |
|---|---|
| Size | 5.9 MB |
| Tags | Python 3 macOS 10.15+ universal2 (ARM64, x86-64) |
|
SHA-256 checksum How to use checksums |
c865df2076b3c7863b51e4d0121023d6c3d2e60ba2d1df3d6b49c2222f4fbb72
|
|
BLAKE2b-256 checksum How to use checksums |
ee43a59f6f62d097e27bab7cc0b5eb09b757382aa8e7d6641e6f6522e1a5f74d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.0.0 CPython/3.8.12
|
