No project description provided
Project description
llama-rs-python
Unofficial python bindings for llama-rs created with PyO3. 🐍❤️🦀
This package gives access to the basic functionality of the llama-rs project.
GGML converted models can be loaded and executed.
Installation
Simply install it via pip: pip install llama-rs-python
Usage
The package is typehinted for easy usage.
A usage example could look like this:
from llama_rs_python import Model
#load the model
model = Model("path/to/model.bin")
#generate
print(model.generate("The meaning of life is").text)
The package also supports callbacks to get each token as it is generated.
The callback-function also supports canceling the generation by returning a True
value from the pytohn side.
import sys
from llama_rs_python import Model
#load the model
model = Model("path/to/model.bin")
#define the callback
def callback(token:str)->Optional[bool]:
print(token,end="")
sys.stdout.flush()
# (return True here to cancel the generation)
#start generation
model.generate("The meaning of life is",callback=callback)
The configuration of the generation is handled by the GenerationConfig
class.
from llama_rs_python import Model, GenerationConfig
#load the model
model = Model("path/to/model.bin")
#create a config
config = GenerationConfig(top_p=0.9)
#generate
print(model.generate("The meaning of life is",generation_config=config).text)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
llama_rs_python-0.0.1.tar.gz
(10.9 kB
view hashes)
Built Distributions
Close
Hashes for llama_rs_python-0.0.1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 34fe85708f07b879604b3066f164fd42a1de1f587843c2c506a5f4d24fb5752f |
|
MD5 | 486a2f75797e1ac69083e60039cddbc8 |
|
BLAKE2b-256 | 43e03d7a054e6cc9b2bdf66084b0cab9c904d72d9a42322f0a3e7c0f2e3cdf2d |
Close
Hashes for llama_rs_python-0.0.1-pp39-pypy39_pp73-manylinux_2_5_i686.manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b2da589fd2be50b928c428a4522249df4532c9ed48057438bb6523f554e17e2c |
|
MD5 | 31208c4b0984449fc2e30004288205dd |
|
BLAKE2b-256 | 392305941fab4b3da03dd0cf0f59606929fec371eed9da12f05a2032e0efd247 |
Close
Hashes for llama_rs_python-0.0.1-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 231083c572a4971c900cd8f0625435cdb859ebb60d6917c3a59a8a53059237ae |
|
MD5 | 305bc7b4e0c91a83218d3819f0edc5de |
|
BLAKE2b-256 | 38859d9c9f3ef6192163d8290e01088cb6f73ad77e2a5c35003ab1eb966d0082 |
Close
Hashes for llama_rs_python-0.0.1-pp38-pypy38_pp73-manylinux_2_5_i686.manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a86b55d4ac313e74f4b0ec740cbb781cc8ccbb39c7c7942d7036e0a5f97563bd |
|
MD5 | efeecad5fb9939c8d3c0349152d99533 |
|
BLAKE2b-256 | cf5be2aa6939b770616b234a04cbb4f0af3e80280a73125869ebc6f4e857d40d |
Close
Hashes for llama_rs_python-0.0.1-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d5e59426fa4b66c4ee78e351f87856a77121f5c5f62aa7549c75a4b1136302d4 |
|
MD5 | dc721686b42015070531c90aea9e2b0d |
|
BLAKE2b-256 | 1a2d134dd6789077513eb93b6c4441dd557c1f241e89b93a46485a708dca97f6 |
Close
Hashes for llama_rs_python-0.0.1-pp37-pypy37_pp73-manylinux_2_5_i686.manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d7ed763b6228f9a898873e1780e5d25eadc6a7999ae31e9a52d445211e19a327 |
|
MD5 | bac61f6d66a07c8b5bac6c4a4be91b3a |
|
BLAKE2b-256 | 1c9dd96716cf35181c742cc2a40828401870909f62ac82a9203c3fa5691d50a0 |
Close
Hashes for llama_rs_python-0.0.1-cp311-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 306f2822c80532caec46e90a73b966763c614efe31f42ba67d8f361093f481ca |
|
MD5 | 8a52841c9b1e0241bd6dd4fcd6e8abcd |
|
BLAKE2b-256 | dbe7ab74d04d143c79b054e60bda0319ae72f13a38eeb0afe50544d970f1206b |
Close
Hashes for llama_rs_python-0.0.1-cp311-none-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5de0fa618ceca089cd1718a34bcc0b31a8cad6151de37a733aa91d29a78924d2 |
|
MD5 | 214db19cc0e5a0c7e883e8f75a9fc071 |
|
BLAKE2b-256 | c81b0e1a7bcf16b7b06dd1461477feacf8b831a0a64999d201809373fafed49a |
Close
Hashes for llama_rs_python-0.0.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dfdfb631d5f5d9960b0ca88cefd3141546d3ccac895e8762f794653d08f91424 |
|
MD5 | 2abc241fd80bde0badd7ff016ad0da07 |
|
BLAKE2b-256 | 4e7bbeb3d6ca07eaacf8a0f0e99902ef8dea227c613a03128f24e806c22fbc18 |
Close
Hashes for llama_rs_python-0.0.1-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 95a2d0f7bee4123044efb4ba915ecb7d1723328b370b3f94b386fe0597a73d08 |
|
MD5 | 183008e9c2b4f58b6ae43ac49ccbd426 |
|
BLAKE2b-256 | 952eaf3cdbc07235057c1fb12f1ab51c7f8911225fd4a74525c04adfe2a0612f |
Close
Hashes for llama_rs_python-0.0.1-cp311-cp311-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | aa53ecb32c3c9fd1c737b599c3c673b0a1338d040d3d6f436f449a5166536dfb |
|
MD5 | 5fdd3829b381b84a9c79482fb206319f |
|
BLAKE2b-256 | 2a92d2b7acbba47e2f7423d3e81871b2718a7e70f8ceb57e234cc8de4efbfb9e |
Close
Hashes for llama_rs_python-0.0.1-cp311-cp311-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fa5ce20ef2a6bf13f86ccba26a920222b1fe471f7709dcfe950ad0cf162e90b2 |
|
MD5 | 25fb4dcce54ed91c90b01828ff899966 |
|
BLAKE2b-256 | de75d4f65fc63048b80b7adaf93187552c2afed408a04956adc35738ffb9f876 |
Close
Hashes for llama_rs_python-0.0.1-cp310-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 881afd383c07e09e52c4c69a62547ca919ca433cf1282e9134217fab230ba766 |
|
MD5 | 76d3d8e32af5d300f65eff58d265ddab |
|
BLAKE2b-256 | b5c04742f739b2f28ca55149875455694653a9720edab097ed7673a15734e9ff |
Close
Hashes for llama_rs_python-0.0.1-cp310-none-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bcd40eec3e59f3c897cf4e255cd8190ad7f636bece75272fd3f1822bb3578939 |
|
MD5 | b89e909d177e80080ab19961c06a7e8a |
|
BLAKE2b-256 | 685753707fadc90bb257520c039c3d6f606456b28311c0d2c03ceb0e0732c523 |
Close
Hashes for llama_rs_python-0.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 06c142c9bfbcef0e865b0592499da7cfd3d130dee38c95cc31808ee532f74c0f |
|
MD5 | 92f483c8ecf5c883dd9e96fc693a5325 |
|
BLAKE2b-256 | cb1f8a65ee6a72d470a69922ecfafa03edea1f2ca2df63baac08ee1d6b735db9 |
Close
Hashes for llama_rs_python-0.0.1-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8dcc7b384d5005d39ecaf1f276a8e17448360163ec6cdb73a33373161f00a130 |
|
MD5 | 939e85bbb3d2921c302f9cffbfa8fa99 |
|
BLAKE2b-256 | f8e635a80e8f87c5fd1af138b189bd5bab724c073201883d447ccca15f406437 |
Close
Hashes for llama_rs_python-0.0.1-cp310-cp310-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7de2fa86ce5cbfa0a4cb0ded8e326850e9a63ce62067c91b4b52540c3d421b69 |
|
MD5 | c73dda08fae28679ad976f3af8e1d1f1 |
|
BLAKE2b-256 | e7b3f02c7146a906fe3db75727149ddcb8c0c8e59c0c899159b6d8c1255e4670 |
Close
Hashes for llama_rs_python-0.0.1-cp310-cp310-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bbe67e44d1544ab2e21064305cd3f29e44cc28fecb91e3313de8d4b9a2c685fd |
|
MD5 | a4dc0a4b1a3947fa3935080472970c43 |
|
BLAKE2b-256 | dcf3357a7b54e3f77210f48c0d59d2e5fdca95fe086aba238b8cddadc07e2c1d |
Close
Hashes for llama_rs_python-0.0.1-cp39-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4390c1f4555a387412ce35ebaf33d7613de9f6b11c6f01621a27cdb24646a589 |
|
MD5 | 54ca586f4b03c669eda73b0416177473 |
|
BLAKE2b-256 | 5c5b3c321b56777cb83634aa6f092f2c0476d59c9400f8d437e4fcb7834d9e69 |
Close
Hashes for llama_rs_python-0.0.1-cp39-none-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | acb93eb80c3fe374735dcd4e291e47faaec9c23b135a08b2b434e091c270d4ed |
|
MD5 | 5a8f5fd7be48de9089391211be25d28d |
|
BLAKE2b-256 | 00373ebe9b08f40003c986d6f24f31a567a23c229de2ad36e1e5697a93e62ab0 |
Close
Hashes for llama_rs_python-0.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0774fd2ff3bdbc635ef00cb3259ddf6ae03dc74511d6a9e1870f2f0ec7b4da91 |
|
MD5 | 36bc895e48f7426dd808418080457369 |
|
BLAKE2b-256 | 56920ccf6a65e722e5a3bac597ee08d098c6ffc6328a67074ca7450e5f862ea5 |
Close
Hashes for llama_rs_python-0.0.1-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | eb3c6e58eb1a6169185dc24a05ebfa48b7111a182adcacd284a495c9dbed5dc7 |
|
MD5 | ba85ae5713f1beeb2aa301a7091e11af |
|
BLAKE2b-256 | 5ca18f12c100983923e4d32f29fcbc80a29981e4419a197655d84966ff3aa4b3 |
Close
Hashes for llama_rs_python-0.0.1-cp38-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d1b19a4a952368ebbdda198ed539f366ec6826891abdebdd876bee3c2413c9b8 |
|
MD5 | 77d3c679138172d4f732bedde7fa6914 |
|
BLAKE2b-256 | e6d37fbb75cd708ab07856f34e274901f0f62f60daeb93207077be12104690d0 |
Close
Hashes for llama_rs_python-0.0.1-cp38-none-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 69876a03cd6abebb70fb49e858aba74051d53a6ae0f39a10ee334e309295a454 |
|
MD5 | 35901b52bce16c1249570703d3b0d3eb |
|
BLAKE2b-256 | 53e9f45b4af612a85c1db5817b14236cccee666c2149a324cc5b5c0517b21e80 |
Close
Hashes for llama_rs_python-0.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7e626cc069bc9d55b6871983a03002f918c18a02412621e281cc034df76f45a4 |
|
MD5 | e8a42fa39a94a6d8d2a18014a41459e0 |
|
BLAKE2b-256 | 67886ac649e87c69b5988c00eb10b7bb6529a0b7b4f1f9b4cc351f50ef4b2eb2 |
Close
Hashes for llama_rs_python-0.0.1-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6b8c3801f42d7b9034715f4a79e2edba8487b124a63d4f2a994ca777c061e3b4 |
|
MD5 | 617f9e3294794d90f4210090dd989e39 |
|
BLAKE2b-256 | 10d34091accf8bcbda832f9142a7fbc9fc0678eb6864b9e4ecfe6ec39892ddd5 |
Close
Hashes for llama_rs_python-0.0.1-cp37-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7e10fe07234a85c970187240ab3fa35f93389a9e2e50d3c1fea9c329db7d1228 |
|
MD5 | f2289fd9fec8986cb5a22da6f930c8c1 |
|
BLAKE2b-256 | f6280cd25f5587bfc48d254f29a56fb969f982ec925e895821eb5d1a65e54518 |
Close
Hashes for llama_rs_python-0.0.1-cp37-none-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 83012e002671964a5a4f3cdc7eabefb0414bd962cd64f29824f66c9951afea9c |
|
MD5 | 01571f7b77798c64afb47fbab871284a |
|
BLAKE2b-256 | 57cbd585d5e0e1f5d7b99bd277e8c6e3de12ff298d8b73f8dd67bcebd851c4b0 |
Close
Hashes for llama_rs_python-0.0.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2e26887cb79995d3bb1ce173e9aac38d41a2882a2dc1facbdd873abc87e860a8 |
|
MD5 | a9f048ea41453ef5d447498b2e111e51 |
|
BLAKE2b-256 | 0838aea97d8f568a6d9e95ed48f02d6252c4d73596dc31e0a9231de66051611a |
Close
Hashes for llama_rs_python-0.0.1-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c984b45e8af495f634b3197e247ca63d3418f124e47a1993e65abc3a4deeb3be |
|
MD5 | f5d9dd656ef8276863e1d12bb66f83ea |
|
BLAKE2b-256 | c899aea919249ca04d3019ccae5fa533fbeb267f8ade0dfd4f01306ae7b957ab |