Run language models on consumer hardware.
Project description
A headless Oobabooga wrapper.
pip install ooba
import ooba
path = ooba.download("https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF")
llm = ooba.llm(path)
messages = [{"role": "user", "content": "Hi Mistral."}]
for token in llm.chat(messages):
print(token)
https://github.com/KillianLucas/ooba/assets/63927363/b741f4ee-9dca-4c50-9405-dd99550b2dc8
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
ooba-0.0.22.tar.gz
(20.9 kB
view details)
Built Distribution
ooba-0.0.22-py3-none-any.whl
(24.5 kB
view details)
File details
Details for the file ooba-0.0.22.tar.gz.
File metadata
- Download URL: ooba-0.0.22.tar.gz
- Upload date:
- Size: 20.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.6.1 CPython/3.11.5 Darwin/21.6.0
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 334b4098e84b6e8c605696d787c64a8bf19b1a5d77e972e4e58159a45a7dfa77 |
|
| MD5 | 43b307b3eb635484b3ce4a17f44a7a19 |
|
| BLAKE2b-256 | e60b1442abee384551465a0990aaecd5534554761a818096a01d998448041ba7 |
File details
Details for the file ooba-0.0.22-py3-none-any.whl.
File metadata
- Download URL: ooba-0.0.22-py3-none-any.whl
- Upload date:
- Size: 24.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.6.1 CPython/3.11.5 Darwin/21.6.0
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 92aaa1534af1ef9d27cae781941448d4c47ba9b7b23a8cc8c92cee5a5e61e185 |
|
| MD5 | 17b2285ef86ccd805c3fc940c9fd257b |
|
| BLAKE2b-256 | 6fb8e72e1cdf34155c43ec98743688bc73b6adf8dafedc5afced514f08983f4a |
