Pre-release

This release is a pre-release and may not be stable for production use.

llm-chat-completions-server

PyPI Changelog Tests License

LLM plugin to serve an OpenAI Chat Completions API endpoint

Installation

Install this plugin in the same environment as LLM.

llm install llm-chat-completions-server

Usage

Start the server:

llm chat-completions-server

It listens on http://127.0.0.1:8002 by default. Use -p or --port to select a different port:

llm chat-completions-server --port 9000

For development, --reload restarts the server when Python files change:

llm chat-completions-server --reload

The server does not require an API token. Models may still need credentials configured on the server using the usual llm keys set commands.

List models

GET /v1/models lists the models registered with LLM that provide an async implementation. Sync-only models are not served.

curl http://127.0.0.1:8002/v1/models

Create a chat completion

POST /v1/chat/completions implements the classic OpenAI Chat Completions request and response format using LLM's async model API.

curl -s http://127.0.0.1:8002/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "gemini/gemini-2.5-flash",
    "messages": [
      {"role": "user", "content": "Tell me a short joke about pelicans"}
    ],
    "stream": false
  }'

Streaming responses use server-sent events and finish with data: [DONE]:

curl -N http://127.0.0.1:8002/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "gemini/gemini-2.5-flash",
    "messages": [
      {"role": "user", "content": "Count to three"}
    ],
    "stream": true,
    "stream_options": {"include_usage": true}
  }'

The endpoint accepts system, developer, user, assistant and tool messages, including image URL content and function tool calls. Sampling options are passed through when they are supported by the selected LLM model. Only n=1 is currently supported.

Completed responses—including streamed responses—are written to LLM's normal logs.db. This populates both the legacy response tables and the current content-addressed message and turn tables. The global llm logs off setting is respected.

Development

This checkout is configured to use the editable in-development LLM checkout at ~/dev/llm. Run the tests with:

cd llm-chat-completions-server
uv run pytest

To run the server with additional model plugins under development:

uv run \
  --with-editable . \
  --with-editable ~/dev/llm \
  --with-editable ~/dev/ecosystem/llm-gemini \
  --with-editable ~/dev/ecosystem/llm-anthropic \
  --with-editable ~/dev/ecosystem/llm-apple-foundation \
  llm chat-completions-server --reload

Project links

Data verified by PyPI on Jul 30, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for simonw from gravatar.com simonw

Credits

Author: Simon Willison

GitHub Statistics

Data verified by PyPI on Jul 30, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

Apache-2.0
View SPDX License List

Requires

Python >=3.10

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

llm_chat_completions_server-0.1a0.tar.gz (15.0 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

llm_chat_completions_server-0.1a0-py3-none-any.whl (12.7 kB view details)

Uploaded Python 3

File details

Details for the file llm_chat_completions_server-0.1a0.tar.gz.

File metadata

File hashes

Hashes for llm_chat_completions_server-0.1a0.tar.gz
Algorithm Hash digest
SHA256 3c70ef5fab6ef9cf3330b4c335f0650f2daefc7bf12e2447dbb657ff9e6efd23
MD5 0278dfa91ba726cbd788ba3bd9175094
BLAKE2b-256 368a10251f5d45e64c445265e742f880772df483d19b5e7cad717aa7a70a5de0

See more details on using hashes here.

Provenance

The following attestation bundles were made for llm_chat_completions_server-0.1a0.tar.gz:

Publisher: publish.yml on simonw/llm-chat-completions-server

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file llm_chat_completions_server-0.1a0-py3-none-any.whl.

File metadata

File hashes

Hashes for llm_chat_completions_server-0.1a0-py3-none-any.whl
Algorithm Hash digest
SHA256 46a864d8a152019cf105765c5b18429fadfdc9061cd1cab38f537512a4b8ad92
MD5 7350e3311435ced70c169ef7364f17df
BLAKE2b-256 9a7823c0334e9b006c7057793cf7e0111745e1e9abb3702448afaad976ad46f6

See more details on using hashes here.

Provenance

The following attestation bundles were made for llm_chat_completions_server-0.1a0-py3-none-any.whl:

Publisher: publish.yml on simonw/llm-chat-completions-server

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page