llm-horde 0.5

Plugin for LLM adding models hosted on AI Horde

llm-horde

LLM plugin for models hosted by AI Horde

Installation

First, install the LLM command-line utility.

Now install this plugin in the same environment as LLM.

llm install https://github.com/spinagon/llm-horde/archive/refs/heads/main.zip

Configuration

First Register an account which will generate for you an API key. Store that key somewhere.

• If you do not want to register, you can use '0000000000' as api_key to connect anonymously. If you don't set your key, it will be used automatically. However anonymous accounts have the lowest priority when there's too many concurrent requests!
• To increase your priority you will need a unique API key and then to increase your Kudos. Read how Kudos are working.

You can set that as an environment variable called AIHORDE_KEY, or add it to the llm set of saved keys using:

llm keys set aihorde

Enter key: <paste key here>

Usage

To list available models, run:

llm models list

You should see a list that looks something like this:

AI Horde: horde/Henk717/airochronos-33B
AI Horde: horde/tgi-fp16-8k/Xwin-LM/Xwin-LM-70B-V0.1
AI Horde: horde/Gryphe/MythoMax-L2-13b

To run a prompt against a model, pass its full model ID to the -m option, like this:

llm -m horde/Henk717/airochronos-33B "Five spooky names for a pet vampire bat"

You can set a shorter alias for a model using the llm aliases command like so:

llm aliases set airochronos horde/Henk717/airochronos-33B

Now you can prompt it using:

cat llm_horde.py | llm -m airochronos -s 'write some pytest tests for this'

If you use "horde" as a model name, you can select one or more models matching a regex pattern using "pattern" option:

llm -m horde -o pattern "13B" "What is the capital of Lebanon?"

You can change the maximum length of returned text by using "max_tokens" option (80 by default), most workers will not :

llm -m horde -o pattern mythomax -s 'Write a sonnet' -o max_tokens 120

If you want to continue generating text without new prompt, try:

llm -c ""

To start interactive chat, type

llm chat -m horde -o pattern llama2