Simple large language model app.
Project description
arey
Arey (अरे, sanskrit) — ind. Interjection of calling.
Arey is a simple large language model app.
🤖 Use any llama.cpp or ollama model.
💬 Chat with your favorite local models. CPU friendly 🍀
🙋 Ask the AI model with a single command.
📋 Supercharged in-context learning workflow ❤️ Edit your prompt in any editor
and arey
will generate a completion on save.
🔓 No telemetry, no internet, nothing to sell. Dedicated to the public domain.
🚧 Much more to come... See Roadmap below.
https://github.com/codito/arey/assets/28766/6b886e49-6124-4256-84d9-20449c783a34
Installation
# Install pipx if needed: `pip install pipx`
# Ensure ~/.local/bin is available in system PATH
pipx install arey
Please use WSL for Windows installation. Troubleshooting notes are here.
Usage
❯ arey --help
Usage: arey [OPTIONS] COMMAND [ARGS]...
Arey - a simple large language model app.
Options:
-v, --verbose BOOLEAN Show verbose logs.
--help Show this message and exit.
Commands:
ask Run an instruction and generate response.
chat Chat with an AI model.
play Watch FILE for model, prompt and generate response on edit.
On the first run, arey
will create a configuration file in following location:
~/.config/arey/arey.yml
for Linux or Mac systems.~/.arey/arey.yml
for Windows.
Please update the models
section in the config yml to your local model path.
1. Ask Arey something!
arey ask "Who is Seneca? Tell me one of his teachings"
2. Chat with Arey
arey chat
3. Run Arey in play mode
Use to fine-tune a prompt in your editor while arey
keeps completing your prompt on every save.
❯ arey play /tmp/arey_playzl9igj3d.md
Welcome to arey play! Edit the play file below in your favorite editor and I'll generate a
response for you. Use `Ctrl+C` to abort play session.
Watching `/tmp/arey_playzl9igj3d.md` for changes...
───────────────────────────────────── 2024-01-21 17:20:01 ──────────────────────────────────────
✓ Model loaded. 0.57s.
Life is short because it passes by quickly and can end at any moment. We should make the most of
our time here on earth and live a virtuous life according to stoicism.
◼ Canceled.
Watching `/tmp/arey_playzl9igj3d.md` for changes...
Development
# Install arey locally in editable mode.
> pip install -e .
> pip install -e .\[test\] # optional, if you wish to run tests
# Install with samples dependency if you wish to run them
> pip install -e .\[samples\]
With OPENBLAS, loading time for models is much smaller and inference is about
4-5% faster. Here's how to install llama-cpp-python
with OPENBLAS support:
> pip uninstall llama-cpp-python
> CMAKE_ARGS="-DLLAMA_BLAS=ON -DLLAMA_BLAS_VENDOR=OpenBLAS" FORCE_CMAKE=1 pip install llama-cpp-python --force-reinstall --upgrade --verbose
If you've a GPU, try the following installation instead.
> pip uninstall llama-cpp-python
> CMAKE_ARGS="-DLLAMA_CUBLAS=ON" FORCE_CMAKE=1 pip install llama-cpp-python --force-reinstall --upgrade --verbose
Roadmap
- Chat and task modes for interactive or batch queries
- Define new tasks with only a prompt, no code. See docs/samples directory for examples.
- Markdown formatting for chat mode.
- Ollama support.
- Command support in chat. E.g., logs, change model, copy, clear, etc.
- Discover prompts from user directory
- Manage prompts and create new interactively
- Download models and manage them
- Release v0.1
- Add textfx
- Add offline knowledge bases and RAG. See https://library.kiwix.org/#lang=eng
License
Dedicated to the public domain with CC0. We'll be delighted if this tool helps you positively 💖
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.