Skip to main content

llm-strata

An llm plugin that exposes the models served locally by Strata — a one-click local AI runtime for Windows/Linux with an OpenAI-compatible API on localhost — as regular llm models, named strata/<model-id>.

If you don't know either: llm is a CLI for running prompts against any LLM with a local logging database; Strata runs open models (e.g. Qwen3.8-Flash-Next) on your own hardware and serves them at http://127.0.0.1:8080/v1. This plugin lets llm talk to Strata's models like any hosted model.

Install

llm install llm-strata

or from a clone of this repo:

llm install -e .

Usage

The plugin discovers the running Strata server and its API key automatically, so once Strata is serving a model you can just run it. The exact model IDs depend on which models your Strata serves, so list them first:

llm strata models
# strata/qwen3.8-flash-next-iq3_s  (aliases: strata/iq3_s)

llm -m strata/qwen3.8-flash-next-iq3_s "hello"
llm -m strata/qwen3.8-flash-next-iq3_s -o temperature 0.3 -o max_tokens 200 "hello"

Streaming, system prompts, token usage and tool calling (llm -m strata/... --tool ...) all work as usual. Reasoning output from thinking models is surfaced as a separate reasoning part.

Only three model options are exposed: temperature, top_p, max_tokens. Everything else (context size, quantization, chat template) is decided by the Strata runtime.

Discovery

Without any configuration, the plugin looks for a server in this order:

  1. A pin saved with llm strata setup
  2. STRATA_BASE_URL environment variable
  3. *.ready.json files in the Strata runtime directories (e.g. %LOCALAPPDATA%\strata\runtime)
  4. http://127.0.0.1:8080/v1

API keys are looked up in this order:

  1. A pin saved with llm strata setup
  2. STRATA_API_KEY environment variable
  3. An strata key stored via llm keys set strata
  4. serving.apiKey in the Strata state.json
  5. api_key in the Strata native-instances files

Every URL/key pair is verified against GET /v1/models before use, so a stale key never produces a confusing 401 later.

Commands

llm strata models             # list models the plugin found
llm strata status             # endpoint + health details (--json for machine output)
llm strata setup              # auto-detect and pin the current endpoint and key
llm strata setup --url URL --key KEY   # pin explicitly
llm strata forget             # remove the pin, fall back to auto-discovery

llm install llm-strata followed by llm plugins should show the strata plugin with a setup command; if llm -m strata/... says Unknown model, start Strata (or run llm strata models to see why nothing was found).

Alternative: no plugin needed

llm (0.36+) ships a built-in command for any OpenAI-compatible endpoint, so you can talk to Strata without this plugin at all. You need the server URL and its API key. Both are shown in the Strata UI under Serving, and from the CLI:

strata launch --status              # address of the running strata-launch server
strata launch --show-key            # its API key
strata connect opencode --show-key  # URL + key the same way an app would be configured

In the launch monitor, press 7 (the Connect view) and k to reveal the key. The default address is http://127.0.0.1:8080; a local server without an API key (the default) needs no --key at all, while sharing beyond this computer makes Strata generate an sk-strata-… key that you must pass.

These CLI commands report servers started with strata launch; when the desktop app is the one serving, read the URL and key from its UI instead. Use them in the example below.

# list the models Strata is serving (URL/key as shown in the Strata UI;
# through the gateway only the short model aliases are listed)
llm openai endpoint http://127.0.0.1:47831/v1 --key sk-strata-… --models

# one-off prompt, interactive chat, tools...
llm openai endpoint http://127.0.0.1:47831/v1 --key sk-strata-… \
  -m iq3_s --chat

--key also accepts a stored key alias (e.g. llm keys set strata --value KEY, then --key strata).

The difference is what you give up: llm openai endpoint does not log to the llm database and you must pass the URL, key and model ID every time. This plugin registers the models permanently, so llm -m strata/... just works, prompts are logged and replayable (llm logs), conversations persist, and the URL/key are discovered from the Strata runtime automatically — including its rotating keys, which is the part you'd otherwise do by hand.

Development

uv sync
uv run pytest

Metadata

Release files for llm-strata 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for llm-strata 0.1.0
File Size Uploaded Provenance Actions
llm_strata-0.1.0.tar.gz 73.6 kB Without provenance Details for llm_strata-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for llm-strata 0.1.0
File Interpreter ABI Platform Actions
llm_strata-0.1.0-py3-none-any.whl Python 3 none any Details for llm_strata-0.1.0-py3-none-any.whl

Total release size: 86.7 kB

Release files / llm_strata-0.1.0.tar.gz

Download URL llm_strata-0.1.0.tar.gz
Size 73.6 kB
Tags Source
SHA-256 checksum
How to use checksums
e1b84990001ff786222c36762c3924b3d229f222b26903c93bb780c5a3da4c84
BLAKE2b-256 checksum
How to use checksums
579655b2f0ee4625d86ac4ad2c9a9dcb26495bbe763d89af3d3b8373e8209baf
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.7.19

Release files / llm_strata-0.1.0-py3-none-any.whl

Download URL llm_strata-0.1.0-py3-none-any.whl
Size 13.1 kB
Tags Python 3
SHA-256 checksum
How to use checksums
aac01ceb0f2d776e27030635700833d8b3b65e0439c11c03fbb85e62b8ca43d2
BLAKE2b-256 checksum
How to use checksums
2aa5867c0968d1dbb49438c6987674b4bc6e277c8a7ee2215245b82506ae8d8d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.7.19

Security for llm-strata 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page