llm-inception

Run prompts against LLMs hosted by Inception Labs with support for their unique diffusing animation feature.

Installation

Install this plugin in the same environment as LLM.

llm install llm-inception

This plugin uses Rich to display diffusing animations.

Usage

First, obtain an API key for Inception Labs and set it using the llm command-line tool:

llm keys set inception
# Paste your INCEPTION_API_KEY here

Alternatively, the plugin will use the LLM_INCEPTION_KEY environment variable if set.

Available Models

To see a list of available Inception Labs models that llm can use, run:

llm models list

The list of models is fetched from the API and cached. You can refresh this cache by running:

llm inception refresh

To see the detailed JSON information for all cached Inception Labs models:

llm inception models

Running Prompts

Run prompts against Inception Labs models like this:

llm -m inception/mercury-coder-small "Tell me a short story about a brave avocado."

Diffusing Animation

This plugin supports Inception Labs' "diffusing" feature, which shows an animation as the model generates its response. This is enabled by default when using the plugin in an interactive terminal.

Example:

llm chat -m inception/mercury-coder-small
> Write a haiku about a robot falling in love.
(Animation will play here)
Whispering wind chills,
Leaves dance under moonlit sky,
Night's soft embrace holds.
>

You can control this feature using the -o no_diffusion true option (note the underscore):

llm -m inception/mercury-coder-small "Why is the sky blue?" -o no_diffusion true

Other model options like max_tokens can also be set:

llm -m inception/mercury-coder-small "Explain quantum entanglement simply." -o max_tokens 150

Development

To set up this plugin locally, first check out the code. Then create a new virtual environment:

cd llm-inception
python -m venv venv
source venv/bin/activate

Now install the dependencies and test dependencies:

pip install -e '.[test]'

To run the tests (you'll need to create some!):

pytest