Python bindings for the GGML GPT-J Laguage model
Project description
PyGPT-J
- Python bindings for GPT-J ggml language models.
- Almost the same API as pyllamacpp.
Table of contents
Installation
- The easy way is to use the prebuilt wheels
pip install pygptj
- Build from source:
git clone git+https://github.com/abdeladim-s/pygptj.git
CLI
You can run the following simple command line interface to test the package once it is installed:
pygtj path/to/ggml/model
Tutorial
Quick start
from pygptj.model import Model
model = Model(model_path='path/to/gptj/ggml/model')
for token in model.generate("Tell me a joke ?"):
print(token, end='', flush=True)
Interactive Dialogue
You can set up an interactive dialogue by simply keeping the model
variable alive:
from pygptj.model import Model
model = Model(model_path='/path/to/ggml/model')
while True:
try:
prompt = input("You: ", flush=True)
if prompt == '':
continue
print(f"AI:", end='')
for token in model.generate(prompt):
print(f"{token}", end='', flush=True)
print()
except KeyboardInterrupt:
break
Attribute a persona to the language model
The following is an example showing how to "attribute a persona to the language model" :
from pygptj.model import Model
prompt_context = """Act as Bob. Bob is helpful, kind, honest,
and never fails to answer the User's requests immediately and with precision.
User: Nice to meet you Bob!
Bob: Welcome! I'm here to assist you with anything you need. What can I do for you today?
"""
prompt_prefix = "\nUser:"
prompt_suffix = "\nBob:"
model = Model(model_path='/path/to/ggml/model',
prompt_context=prompt_context,
prompt_prefix=prompt_prefix,
prompt_suffix=prompt_suffix)
while True:
try:
prompt = input("User: ")
if prompt == '':
continue
print(f"Bob: ", end='')
for token in model.generate(prompt, antiprompt='User:'):
print(f"{token}", end='', flush=True)
print()
except KeyboardInterrupt:
break
API reference
You can check the API reference documentation for more details.
License
This project is licensed under the MIT License.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
pygptj-2.0.0.tar.gz
(221.3 kB
view hashes)
Built Distributions
Close
Hashes for pygptj-2.0.0-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5bd538c9200315921e6f3d193a1895a134190886aed3c7411e6c66952071ecce |
|
MD5 | 2007fbe456a916a4297d8f978a77c55d |
|
BLAKE2b-256 | d496cbf070f12c04305e22224172bbee3a0deb679c13ff58ba5a7abff2446be0 |
Close
Hashes for pygptj-2.0.0-pp39-pypy39_pp73-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3b6e44fec8d0a9774675f31f730f5063c3bddda5bd81ef21aa5d12e93eb896c0 |
|
MD5 | 1d67f71fb84f909fb982fa1e9db8f971 |
|
BLAKE2b-256 | bf8f07803e70ad05ae95d99e19438d6edafeaac7a7cc0310ef8fcfa6dc66af4b |
Close
Hashes for pygptj-2.0.0-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f383b884ba12a32a713f3ebb4392e0836e182660ff68a61e700b870c79f73cb5 |
|
MD5 | d91a68f53bebb1a10ca88887c47d6f63 |
|
BLAKE2b-256 | f1e512fecca819c96160ee2b405b37ec9ce787d4da4cbac74b5a91c0ba14fdf9 |
Close
Hashes for pygptj-2.0.0-pp38-pypy38_pp73-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 664237f769eebc07fe995a5aa9914266e0db4667b8c7d501a9a25d00c4bcb1a5 |
|
MD5 | 818f18ea2a4f93d97b1cbbd45efdc947 |
|
BLAKE2b-256 | 0d256480ef5a0ffc7fa19f733eef2c04e410a42200efd692321ff8c5debb11a4 |
Close
Hashes for pygptj-2.0.0-cp311-cp311-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fa2197da815600cda100310da625f39549f576399e59019b337cf2a87aab2384 |
|
MD5 | b7cfdeb18c7ba1125f6e9d94d8f3cc27 |
|
BLAKE2b-256 | 4eefed93e92a73477ee70d4512c32b89777b5eeb2e5ebb50e58fbfee80abd377 |
Close
Hashes for pygptj-2.0.0-cp311-cp311-musllinux_1_1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 685028ae79d1d18e0455b38866c3a1e22df13dc102252383cff16b5a5dbec61c |
|
MD5 | 86f2d66d5d402e05c3e82431f12813ba |
|
BLAKE2b-256 | b137b7b0141161c65be6bc54e4aebdfd59295a434cd14d67daf9b01a3b22c0a5 |
Close
Hashes for pygptj-2.0.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | abd9604074fc9d9e963222325ec04c494c11819d2d7eda524670549e247bdf6f |
|
MD5 | 04e40ef9ee364109fd1b39d654a69cd2 |
|
BLAKE2b-256 | 318d0a06db8902311b000ae1ff52ea564c4d58e1122ed65c86890ec6d27aa37f |
Close
Hashes for pygptj-2.0.0-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7385c373fbb1ef4a3d3e4f4088dcc9812f36d11c72f5ce404e1229586698c738 |
|
MD5 | 20468b30d9897428d584c79a29ffba5e |
|
BLAKE2b-256 | 7331703b2685d7f6bf5d978c2a3918ce3dd2c38c8c54416e9d6041ca24742e9c |
Close
Hashes for pygptj-2.0.0-cp310-cp310-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1956524ef315344064f4caa742ea8ef05d092b5951cb427601917a6c7be905c5 |
|
MD5 | 32a6d626e3f1c8b9db4a8c3d9d43f916 |
|
BLAKE2b-256 | bcdabd4eb9deb046991ad3aa8524d23182aaebfc17b681e0edfaf37afc6d32ce |
Close
Hashes for pygptj-2.0.0-cp310-cp310-musllinux_1_1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3b5f2f6efb281a9e07ed0cc28e4cf963815345f2455f0948c7c65103b29d1f50 |
|
MD5 | 2d17c34e2aa3b58b491f6e0fbadc78c4 |
|
BLAKE2b-256 | 993736c67c701b3f009153995aead04e612094c7b405f7c2246e3340771ca2ef |
Close
Hashes for pygptj-2.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6058eb863e007807e5256c293c512fe19079f62c5270a66e6b3c24547182d192 |
|
MD5 | 64a3a86e7805a625052a202d113378bf |
|
BLAKE2b-256 | e8dbf6e00b46dabe0990991a27ffc64290d987d2dffc981c5a867f3cdd422f55 |
Close
Hashes for pygptj-2.0.0-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1eb9f9a5fa3200d3fabf6d95146be7a9d0c656bee9886c24a9421025c63645d4 |
|
MD5 | 5015b2837dd3fa1dfe8097d26a08f4f1 |
|
BLAKE2b-256 | a3c1087a513d41951fbe0a06ff210a194608c99309d04fd9fdde60115c0f5bfd |
Close
Hashes for pygptj-2.0.0-cp310-cp310-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ddebae5d2ea0a0b8d2087a40b88c2e5afca683207a1fd258754a3ccd7628bb60 |
|
MD5 | b0a457e7f31ca6edf94211dd78c5827d |
|
BLAKE2b-256 | be5a4a9cbf575e8c0692d9a34722be47f38e73b379b63334dc16f83c0d2eda7f |
Close
Hashes for pygptj-2.0.0-cp39-cp39-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f69530d016bccade0bc82e52981b81c46abd21c667bdc0caa5c4e7f2cfd90f2e |
|
MD5 | 5947b08235a22a39d73bca4d16913e7b |
|
BLAKE2b-256 | f1a07e7b7613007810c4dd0b593927e821608ac7a9679be372f6f3d4dfb44fd8 |
Close
Hashes for pygptj-2.0.0-cp39-cp39-musllinux_1_1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f27bb0f579ab4ff69f6fe88ad477fb6344522ea996797058e2036c3c968ca19d |
|
MD5 | a51015f5c47d3c1e854d7a21ff060913 |
|
BLAKE2b-256 | 0502f4afec5b438b8f29128c359c5fabc567cd100bca69655a67de189c931e8a |
Close
Hashes for pygptj-2.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2d2ad967e92f9731d22e6bb4f35106d2ce32cb943ae0a50891295dad85cc4090 |
|
MD5 | bfa0a5fcbb5df2df6f2636ad62d87908 |
|
BLAKE2b-256 | a2d0ecf49db9ae3add9b461832d251a1a7b2d001427a836a687931479413a642 |
Close
Hashes for pygptj-2.0.0-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 039ebe65ab66d0d103b050954ec8622d8abbb32f1d5f0a02d184f22133fa71e3 |
|
MD5 | a45c57620dc4689b8f98ea2d521fbd60 |
|
BLAKE2b-256 | 5fe76f4a950e26184c1a2abb8972aacac7ff53d327a8790b772d6861fa45b25f |
Close
Hashes for pygptj-2.0.0-cp38-cp38-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 00e48f288cfc86e7067dbb7ebd8d6be2f9d139f13784ead6e6dc243b3ea59a04 |
|
MD5 | b83fcb79611737626ffdef756e81d8e7 |
|
BLAKE2b-256 | 98c3c953547e11ef4f906e367a48813ac52070f62823b59f432dc45640289eab |
Close
Hashes for pygptj-2.0.0-cp38-cp38-musllinux_1_1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 505837419c7d5f76d2205a620e4c37944523aba78f81b48fbc798b81156d6601 |
|
MD5 | a1247a6d10492d8217a50dc6f6ec2601 |
|
BLAKE2b-256 | 950ee261f3e29dfc75376d641792bd11aee9bed58ead80a5da1d1da67cb798cc |
Close
Hashes for pygptj-2.0.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e9ba1377535ede6e78a6c3f9237a82bc47fcb860ba4de834f018350f3a8c5328 |
|
MD5 | 8fbcda10620950f184352e99a0ad39cc |
|
BLAKE2b-256 | adc5671a058463d7be51aa23ae85de7149ea30fa84f4141dc5e340d2d1c3aa66 |
Close
Hashes for pygptj-2.0.0-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7484071a03f78688a19646dbac5b04bad866e72ae21ab86a929c0faa39bdf150 |
|
MD5 | ddb9ce98e023df6d3e8070bb711d6811 |
|
BLAKE2b-256 | 133e9b3dac4883b12b062fe57df9b50e25be524fb21b961723fbf497fe2619d0 |