llm-llama-server 0.1

Interact with llama-server models

llm-llama-server

Interact with llama-server models

Installation

Install this plugin in the same environment as LLM.

llm install llm-llama-server

Usage

You'll need to be running a llama-server on port 8080 to use this plugin.

To access a regular model from LLM, use the llama-server model:

llm -m llama-server "say hi"

For vision models, use llama-server-vision:

llm -m llama-server-vision describe -a path/to/image.png

Development

To set up this plugin locally, first checkout the code. Then create a new virtual environment:

cd llm-llama-server
python -m venv venv
source venv/bin/activate

Now install the dependencies and test dependencies:

python -m pip install -e '.[test]'

To run the tests:

python -m pytest