Unofficial python bindings for llm-rs. 🐍❤️🦀
Project description
llm-rs-python: Python Bindings for Rust's llm Library
Welcome to llm-rs
, an unofficial Python interface for the Rust-based llm library, made possible through PyO3. Our package combines the convenience of Python with the performance of Rust to offer an efficient tool for your machine learning projects. 🐍❤️🦀
With llm-rs
, you can operate a variety of Large Language Models (LLMs) including LLama and GPT-NeoX directly on your CPU.
For a detailed overview of all the supported architectures, visit the llm project page.
Installation
Simply install it via pip: pip install llm-rs
Usage
The package is type-hinted for easy usage.
A Llama model can be run like this:
from llm_rs import Llama
#load the model
model = Llama("path/to/model.bin")
#generate
print(model.generate("The meaning of life is"))
Documentation
For in-depth information on customizing the loading and generation processes, refer to our detailed documentation.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for llm_rs-0.2.1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e0379e5bcd66c2929d544410ec3aebd4bff78d1aa664838602e71d6492d0ed80 |
|
MD5 | d1abbd70316a6e8dc3064feb68a03b26 |
|
BLAKE2b-256 | 0cf600eaf25a17f11b5505ac9245f62b0625e9e4b663ef6d3a87989a507c0be5 |
Hashes for llm_rs-0.2.1-pp39-pypy39_pp73-manylinux_2_5_i686.manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b615dcf9bcb12d8db6d99ca9f6fef386c281e47f09dcbd06b586cccafbf4529d |
|
MD5 | ee168c5e9e2eded72a05a2bdf91b8589 |
|
BLAKE2b-256 | 5d351d8fd767dd370296a1263ee4d2b2c8d09fce1d2aa77fd4367d015b9ac529 |
Hashes for llm_rs-0.2.1-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f661fbf823bdde6f1699635552f01395d7d3ffa14a93a429cdb65b48e927e824 |
|
MD5 | f91e032b9a158c95debe137e6115c8c1 |
|
BLAKE2b-256 | 35e9692c168e826ee2cadf600c12ad6d9b52a9ae5c818e182f15b6e9c9a62703 |
Hashes for llm_rs-0.2.1-pp38-pypy38_pp73-manylinux_2_5_i686.manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d12aca936b63ec239fbc91ae40f4baed3743a1f6c0ae21c6d5835b1b018e9f62 |
|
MD5 | dcd1a0d17630d94bc2b309bec3e46e29 |
|
BLAKE2b-256 | 233760e076548bcd63bad7eb66be576dd7a5026757c905de7fe2a49d02e374ba |
Hashes for llm_rs-0.2.1-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e752064764643895e6c7b5f463558602b9d5eeb0387e44ff5631d33203e60628 |
|
MD5 | aa8aa6a91b85b6b45317e8ee0021b630 |
|
BLAKE2b-256 | e2e7641b688463fe3069f4f016fd09fcf561d49d57648b44092e37e5992fc351 |
Hashes for llm_rs-0.2.1-pp37-pypy37_pp73-manylinux_2_5_i686.manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d8e11c0fbdaf09e3d14e03afae6629bce94ae0af1e1f5a7177e1f4b6fc4016f6 |
|
MD5 | 0c8de55286a75e9cfa01d79cd96ca5fd |
|
BLAKE2b-256 | 88470b84c0250f36a93a23d75ec3415b412352ff33a0846748f8ced803f0d791 |
Hashes for llm_rs-0.2.1-cp311-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f4c8c3d27c6e0c3c83c25d39f26261e3e56050b0a91b40928c3b8b8518c7e447 |
|
MD5 | e37b7e2a387be6d3e22be9fd1c99b82a |
|
BLAKE2b-256 | 1ce1e59495a426f25dfeed3c2ff0452bc623e1c9c02e1bbde7366efee7f38f38 |
Hashes for llm_rs-0.2.1-cp311-none-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 26fefc607e2218d0b259f2611082193fb436b3c48642cef2c4a1f2849d915ff8 |
|
MD5 | 9c90ba4cd94718f53c3fb3dad98b7332 |
|
BLAKE2b-256 | d0d41902c6771deab42d72c747702d9490f0758958945ed1ea8dee296687741a |
Hashes for llm_rs-0.2.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0aa8004b72f028f40d5425bef69af2948dc8e9f15cbcf426cdc7558e0ac21cde |
|
MD5 | 334c000d8f94a53f7e7974700ee6e182 |
|
BLAKE2b-256 | d0bb08b21801b63ebeb170e9d7707da86011890e662651cddc308d46bb64ca0f |
Hashes for llm_rs-0.2.1-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 28655672c09ac076e9677eadbde841b32aa1ce48b7d9b8cfe8e75edd6e5337f1 |
|
MD5 | d5eb90ac816e956eae991d9d7916bf03 |
|
BLAKE2b-256 | ed606e93fc79ab5cf5df5f28becc24f41ec5f21abb5d57f92d20f5c1127c2095 |
Hashes for llm_rs-0.2.1-cp311-cp311-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 227f687f1d408c6e8362bad73363a894ca2311b1d9807d1e3494088674b45fd5 |
|
MD5 | 9a481f9c16bf0f9bcabe6cad9b361a90 |
|
BLAKE2b-256 | 5dd9462dfae691acacd2a7d61d31f3beaec9f2d32e1fef234022999dbad1791c |
Hashes for llm_rs-0.2.1-cp311-cp311-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 68d0d74e4c503393adafc3697c4cdfae710f9a18c82534223c2ae2ad13d0cdcb |
|
MD5 | 7eb533d7ae2217c58c49df09d18c46bc |
|
BLAKE2b-256 | 4ebc0871386ccb9e417344dd83ee452d91e3d1802654455f73473eac7421f286 |
Hashes for llm_rs-0.2.1-cp310-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5b46beb7680fecff71a68026fff3d0c7dbe21d5d82eecc483a8a248e39a181d1 |
|
MD5 | dd31c8fbb7119b39c91b6a544603312d |
|
BLAKE2b-256 | f1089bd574d192e0ce5d1329e2d326b8016056c34d3882fe187a81605e2222b9 |
Hashes for llm_rs-0.2.1-cp310-none-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0b4d367165fc85bc75e6801728aebf1c9ba9afd320f0be0a31726fbd940ec74e |
|
MD5 | 2e6b1256e98ad3ca1629ed2395d6b013 |
|
BLAKE2b-256 | cf950740d08e34f357f884b0917f4ff9c2ad2526a5b649f1f396759a79b232ae |
Hashes for llm_rs-0.2.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 563182fd51023f1d7898b509ed39e009a19c0ce352d8c410360154203eef68e2 |
|
MD5 | c352cab84a4f8289d7e3a12233230b14 |
|
BLAKE2b-256 | 21ceca0ee563556bdc0dc60e69c3aaceab030a35978c6c57f3838211577224bf |
Hashes for llm_rs-0.2.1-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ad355567f6856b5b84f51f1c81e80a96b6914504b7691023deb8f0eccf321b95 |
|
MD5 | f0557179747f8c46d0a2d336bb67eae1 |
|
BLAKE2b-256 | 9cb7127e7da62e9178b07070ae339729d38a79d7b3041d86f4d5613915ea224c |
Hashes for llm_rs-0.2.1-cp310-cp310-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cec8f95d25922d1c17da29e6b367a8325fc2d73a01a765790fef25032456b413 |
|
MD5 | 6f2bef4ce64cca1971e084fa4e7d7896 |
|
BLAKE2b-256 | 05e346ec0e0966af3c8678814c3ed42d0f3167948768a1f0c7cafd735d45160f |
Hashes for llm_rs-0.2.1-cp310-cp310-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 93e38ec5a4f34ee1cdacd5c93828ae2212eb946d5f6f451b649647906aa7e362 |
|
MD5 | ede00a93d97cfb1f3dcbc0845284d612 |
|
BLAKE2b-256 | cf37ee20af9ae10df951c6ac0b1860d469abb8a5eadb0ff87362ec084011e992 |
Hashes for llm_rs-0.2.1-cp39-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ce51c2f9ce68427f5d736803b17a38d6db879d2fb5cbee994a920220f102fd91 |
|
MD5 | c09c07ab9a90de8302e987bda0108e77 |
|
BLAKE2b-256 | e2d7c6e24a2c2ad40bf917aa8f61cef92fe5c08e1b9532fa978bf541cbb349b8 |
Hashes for llm_rs-0.2.1-cp39-none-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 99c324c5ff6e74c93e57a401a81a60039b7b935230d20fac2a24dfb0ac6eff79 |
|
MD5 | a5f8b7de5cac7e60eed1d033fb47aff4 |
|
BLAKE2b-256 | 70de468e22e402866616512a846692ce88546d55d446c5d1457f4f5c2a6033dd |
Hashes for llm_rs-0.2.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f977e08bffcac035607cd83255f24e6a1e117fecb2ff0dfdf6527547bccf7db7 |
|
MD5 | 7bcc06e492e573977d3a9bc2930f8648 |
|
BLAKE2b-256 | a2eeb999c4e72f36dca4fa3e5e1e057180717b4239db32a4cc92625f4160ff2c |
Hashes for llm_rs-0.2.1-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 06131b6ff46eda8288214dedbb184cde94cb285bcb97ae6834cb2d669deb955a |
|
MD5 | 1e9e5acd5581a16c3fe36c03d5124f4d |
|
BLAKE2b-256 | 79a820d95da68849a40ff6ae8a247f5cdcff729d58e6bd970caacadb4c6b79ae |
Hashes for llm_rs-0.2.1-cp38-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c5f2a14db1413ac70f773ba9a9b0461a78952f868675b4e1f6b767d2a4011d00 |
|
MD5 | c279bb17019648cd6e1420c6b8d45dec |
|
BLAKE2b-256 | 2aad6fe33abebe41cd548fa76ac5ca6b81b5ca238f344b27e9e30f0cdb1e4c24 |
Hashes for llm_rs-0.2.1-cp38-none-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 758cb19b0b2b9fd2605ac3f34edb67c260519979dd6670514faf0289697083a1 |
|
MD5 | 9accbd0f05c462cd6131986eed4a93e0 |
|
BLAKE2b-256 | c7b84e0a6f98d1bbe128b065a3b8b67c5b024d06548005e5a04498f3ff84e7b9 |
Hashes for llm_rs-0.2.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d77e5713a4e20221963f3488fa8868530074f749345842dd177bdec9b432f612 |
|
MD5 | 1510082b1d53947d4731f7b86548a4af |
|
BLAKE2b-256 | 1a74d18f0dd89467f5ee6a8a5ea47649f97480e051bb0af582709abca7d7ab98 |
Hashes for llm_rs-0.2.1-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ecc2d67d428b1a0196d67dfa84227455186acc13e4e82b79144300e91b3bfb36 |
|
MD5 | c0545b705ae47a944131db58465664eb |
|
BLAKE2b-256 | ab0f95749bae6cd8bed3be06f175df8a67871c54ccc3cce65cdaf4f0fd5c8027 |
Hashes for llm_rs-0.2.1-cp37-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c7fe85adcdf3d33cb9a5522b81293b64f3787a5e058799307b8ef6b5dc7800d0 |
|
MD5 | 8935dc988760d76f1783aad4e120b1c1 |
|
BLAKE2b-256 | 24c052f19435f945756379fae58c93696658fb5d594bf897087bdb38481c058d |
Hashes for llm_rs-0.2.1-cp37-none-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c134a9faa5e8c10ca5945d7f3d97c34ce0ce303e32259d57862e5518dd24c87e |
|
MD5 | 0dc2a566fdf5445780daa191332232ed |
|
BLAKE2b-256 | 6c8b6a8e17f59cec1288a2dd70a508f483eb35714187f4b97121f8ea0c01cbd5 |
Hashes for llm_rs-0.2.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6f6b7fbeb4383f5b0ed9adb6cd625a72e00ed0bcf8e86a68d9135ea065399bab |
|
MD5 | fb24b113b8478a311a59b521ffa0f0c8 |
|
BLAKE2b-256 | 1d59ab58f1b3d7f5fe334300115c4dd94496acb550efd5fc6cceed063c47fdbe |
Hashes for llm_rs-0.2.1-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9b275771d7ddd6c5f7419f22a53540040a9e4ecbd1465f339e56706161755307 |
|
MD5 | d6742b247d46976b6aa72978f2aeea89 |
|
BLAKE2b-256 | 86e1a844cc7bc0be3ecf5ac00109b2ec8441c1061ffaf8c6e4e71ff814d3f4b5 |