Run language models on consumer hardware.
Project description
A headless Oobabooga wrapper.
pip install ooba
import ooba
path = ooba.download("https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF")
llm = ooba.llm(path)
messages = [{"role": "user", "content": "Hi Mistral."}]
for token in llm.chat(messages):
print(token)
https://github.com/KillianLucas/ooba/assets/63927363/b741f4ee-9dca-4c50-9405-dd99550b2dc8
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
ooba-0.0.6.tar.gz
(19.1 kB
view hashes)
Built Distribution
ooba-0.0.6-py3-none-any.whl
(22.4 kB
view hashes)