llm-lmstudio

This is a plugin for Simon Willison's LLM command-line utility that lets you talk to models running on a local LMStudio server.

Version 0.3.0 and later require the llm Python package 0.32 or newer. This dependency provides structured messages, response parts, and resumable tool chains.

The plugin also requires LM Studio 0.4.0 or newer. This version introduced the native v1 REST API.

Installation

Make sure you have llm installed, then install this plugin from PyPI:

uv pip install llm-lmstudio

Or, to install the latest development version directly from GitHub:

llm install -U git+https://github.com/agustif/llm-lmstudio.git

Alternatively, llm install llm-lmstudio will also find and install the plugin.

Usage

First, start the LM Studio server. The API usually runs at http://localhost:1234.

The plugin finds downloaded models through the LM Studio API. A model does not need to be loaded first. You can check this using the llm models | grep lmstudio command:

llm models list

You should see your LMStudio models listed, prefixed with lmstudio/ (e.g., lmstudio/llava-v1.5-7b).

To run a prompt against a model:

# Replace 'your-model-id' with the actual ID shown in 'llm models list'
# e.g., llm -m lmstudio/llava-v1.5-7b "Describe this image"
llm -m lmstudio/your-model-id -o temperature 0.7 -o max_tokens 100 "Tell me a joke"

To start an interactive chat session:

llm chat -m lmstudio/your-model-id

You can exit the chat by typing exit or quit.

Vision Model Support

The plugin supports vision-language models (VLMs).

• When using a VLM, you can attach images using the standard llm attachment syntax:

  llm chat -m lmstudio/your-vlm-id -a path/to/image.png "Describe this image"
  

  Or for a single prompt:

  llm -m lmstudio/your-vlm-id -a path/to/image.png "What is in this picture?"
  

• The plugin will encode the image and send it to the model.
• This feature's success depends on the specific VLM, its configuration in LM Studio, and LM Studio's API correctly handling image data.
• Models that support vision may have a 👁️ (eye icon) in their display_suffix when inspected via llm inspect -m lmstudio/your-vlm-id, though this may not always render in llm models list.

Tool Support

Some models loaded in LMStudio can call tools. The plugin will surface those tool invocations and their results when interacting with such models.

Example:

$ llm --tool llm_version "What version of LLM is this?" --td

For more information about tool calling support consult the llm documentation on tools.

Embedding Models

If you have embedding models loaded in LMStudio (their names usually contain "embed"), the plugin will register them too. You can list them with:

llm embed-models

You should see models like lmstudio/text-embedding-nomic-embed-text-v1.5@f16.

To generate embeddings for text using one of these models:

llm embed -m lmstudio/your-embedding-model-id -c "This is the text to embed"

Configuration

The plugin connects to the LMStudio server API. By default, it tries http://localhost:1234.

You can configure the server URL(s) using the LMSTUDIO_API_BASE environment variable.

• For a single server:

  export LMSTUDIO_API_BASE="http://your-server-address:port"
  

• For multiple servers (the plugin will try them in order):

  export LMSTUDIO_API_BASE="http://server1:1234,http://server2:5678,https://server3:custom_port"
  

The variable accepts one or more http[s]://host:port values, separated by commas (spaces around commas are optional). The plugin automatically attempts to append /v1 or /api/v0 to the determined base URL(s) as needed when probing the server.

Model Options

You can pass generation options supported by the LMStudio API (like temperature, max_tokens, top_p, stop) using the -o flag:

llm -m lmstudio/your-model-id -o temperature 0.7 -o max_tokens 100 "Tell me a joke"

Automatic Model Loading

If a selected model is not loaded, the plugin loads it through POST /api/v1/models/load. The plugin does not require the lms CLI.

The endpoint returns after the load operation finishes. Set LLM_LMSTUDIO_DEBUG=1 to show load timing and instance details.

The plugin requires LM Studio 0.4.0 or newer. LM Studio 0.4.0 introduced the native v1 REST API at /api/v1/*.

Development

To set up this plugin for development:

1. Clone the repository.
2. Run tests uv run pytest

Or do the classic complicated dance:

1. Create a virtual environment: python -m venv .venv
2. Activate it: source .venv/bin/activate
3. Install dependencies, including dev dependencies: pip install -e . --group dev
4. Run tests: pytest

Live acceptance verification

manual-testing.md is an executable Showboat document. It verifies the plugin against a live LM Studio server with the documented GGUF, MLX, embedding, and vision models.

uvx showboat verify manual-testing.md

This command is slower than the unit tests. It also unloads and reloads models in LM Studio.

The asynchronous tests in tests/test_llm_lmstudio_async.py (among others) use pytest-vcr to record and replay HTTP interactions with the LM Studio server. To record new cassettes:

1. Ensure LM Studio is running with the target model(s) loaded (e.g., llava-v1.5-7b).
2. Temporarily set record_mode='all' in the @pytest.mark.vcr decorator for the relevant tests in tests/test_llm_lmstudio_async.py.
3. Run pytest -v -s tests/test_llm_lmstudio_async.py.
4. Cassettes will be saved in tests/cassettes/.
5. Important: Change record_mode back to 'once' after recording.
6. Commit the new/updated cassettes.

Missing features / Known Issues:

• The reliability and capabilities of image support can vary significantly based on the specific VLM and its implementation within LM Studio.
• The plugin does not support authentication for remote LM Studio servers. Community contributions for this feature are welcome.
• Automatic loading does not support LLM_LMSTUDIO_TTL. The /api/v1/models/load endpoint does not accept a TTL parameter.

Future work

• Add server-side tool support through LM Studio's /api/v1/chat or /v1/responses endpoint. The current /v1/chat/completions integration supports custom tools only. Community contributions are welcome.
• Migrate manual-testing.md to an opt-in Pytest acceptance suite that runs against a live LM Studio server. Keep these slower tests out of normal pytest runs.