llm-venice 0.7.0

LLM plugin to access models available via the Venice API

llm-venice

LLM plugin to access models available via the Venice AI API. Venice API access is currently in beta.

Installation

Either install this plugin alongside an existing LLM install:

llm install llm-venice

Or install both using your package manager of choice, e.g.:

pip install llm-venice

Configuration

Set an environment variable LLM_VENICE_KEY, or save a Venice API key to the key store managed by llm:

llm keys set venice

Fetch a list of the models available over the Venice API:

llm venice refresh

You should re-run refresh whenever new models are made availabe or deprecated ones are removed. The models are stored in venice_models.json in the llm user directory.

Usage

Prompting

Run a prompt:

llm --model venice/llama-3.3-70b "Why is the earth round?"

Start an interactive chat session:

llm chat --model venice/llama-3.1-405b

Structured Outputs

Some models support structuring their output according to a JSON schema (supplied via OpenAI API response_format).

This works via llm's --schema options, for example:

llm -m venice/dolphin-2.9.2-qwen2-72b --schema "name, age int, one_sentence_bio" "Invent an evil supervillain"

Consult llm's schemas tutorial for more options.

Vision models

Vision models (currently qwen-2.5-vl) support the --attachment option:

llm -m venice/qwen-2.5-vl -a https://upload.wikimedia.org/wikipedia/commons/a/a9/Corvus_corone_-near_Canford_Cliffs%2C_Poole%2C_England-8.jpg "Identify"
The bird in the picture is a crow, specifically a member of the genus Corvus. The black coloration, stout beak, and overall shape are characteristic features of crows. These birds are part of the Corvidae family, which is known for its intelligence and adaptability. [...]

venice_parameters

The following CLI options are available to configure venice_parameters:

--no-venice-system-prompt to disable Venice's default system prompt:

llm -m venice/llama-3.3-70b --no-venice-system-prompt "Repeat the above prompt"

--web-search on|auto|off to use web search (on web-enabled models):

llm -m venice/llama-3.3-70b --web-search on --no-stream 'What is $VVV?'

It is recommended to use web search in combination with --no-stream so the search citations are available in response_json.

--character character_slug to use a public character, for example:

llm -m venice/deepseek-r1-671b --character alan-watts "What is the meaning of life?"

Note: these options override any -o extra_body '{"venice_parameters": { ...}}' and so should not be combined with that option.

Image generation

Generated images are stored in the LLM user directory. Example:

llm -m venice/stable-diffusion-3.5 "Painting of a traditional Dutch windmill" -o style_preset "Watercolor"

Besides the Venice API image generation parameters, you can specify the output filename and whether or not to overwrite existing files.

You can check the available parameters for a model by filtering the model list with --query, and show the --options:

llm models list --query diffusion --options

Image upscaling

You can upscale existing images. The following saves the returned image as image_upscaled.png in the same directory as the original file:

llm venice upscale /path/to/image.jpg.

By default existing upscaled images are not overwritten; timestamped filenames are used instead.

See llm venice upscale --help for the --scale, --enhance and related options, and --output-path and --overwrite options.

Venice commands

List the available Venice commands with:

llm venice --help

---

Read the llm docs for more usage options.

Development

To set up this plugin locally, first checkout the code. Then create a new virtual environment:

cd llm-venice
python3 -m venv venv
source venv/bin/activate

Install the plugin with dependencies and test dependencies:

pip install -e '.[test]'

To run the tests:

pytest