pypaya-llm-client

One small client for every OpenAI-compatible LLM server — local or hosted — with images and audio as plain message parts.

Most "LLM client" code differs between providers only in the base URL. This library leans into that: one class, preset shorthands for the common servers, and multimodal input handled the way the chat protocol actually works — as content parts in a message list. ~330 lines, one required dependency (openai).

Install

pip install pypaya-llm-client
# with in-memory PIL image support:
pip install pypaya-llm-client[pil]

Quick start

from pypaya_llm_client import LLMClient

# Local model via Ollama
client = LLMClient(base_url="ollama", model="gemma4:latest")
print(client.chat("What is a neural network?").content)

# Same code, hosted via OpenRouter
client = LLMClient(base_url="openrouter", model="qwen/qwen3-vl-30b-a3b-instruct",
                   api_key="sk-or-...")

Preset base URLs (KNOWN_BACKENDS): ollama, lmstudio, vllm, llamacpp, openrouter, openai — or pass any full URL.

Images and audio

Modalities are just content-part types; pass file paths, raw bytes, or PIL images:

reply = client.chat("What text is on the button in the bottom-left?",
                    images=["screenshot.png"])

reply = client.chat("How many beeps do you hear?", audios=["alert.wav"])

Audio requires an audio-capable (omni) model. Note from real testing: an "audio input" flag does not guarantee audio understanding — pick models known to comprehend your kind of audio, and verify with a cheap test of your own.

Multi-turn conversations, mixed modalities

from pypaya_llm_client import LLMClient, build_user_message

client = LLMClient(base_url="openrouter", model="google/gemini-2.5-flash", api_key="...")
messages = [{"role": "system", "content": "You are analyzing a GUI test session."}]

messages.append(build_user_message("The session ID is ALPHA-7. Confirm."))
messages.append({"role": "assistant", "content": client.chat_messages(messages).content})

messages.append(build_user_message("New screen and new alert sound attached. "
                                   "How many beeps, what does the orange button say, "
                                   "and what was the session ID?",
                                   images=[screenshot], audios=[alert_wav_bytes]))
print(client.chat_messages(messages).content)   # e.g. "2, DEPLOY, ALPHA-7"

Metrics for free

Every call returns an LLMResponse with content, model, duration_ms, prompt_tokens, completion_tokens — enough to track latency and cost without extra instrumentation.

Per-call overrides

client.chat("Say OK.", model="qwen/qwen3-vl-8b-instruct", max_tokens=10)

Useful for model ladders (try a cheap model, fall back to a stronger one) without constructing new clients.

API surface

• LLMClient(base_url, model, api_key, temperature, timeout, max_tokens)
• .chat(prompt, *, system, images, audios, model, max_tokens) -> LLMResponse
• .chat_messages(messages, *, model, max_tokens) -> LLMResponse
• .is_available() -> bool — probe the server
• build_user_message(text, images, audios) -> dict — build one OpenAI-format message
• encode_image_base64(path | bytes | PIL.Image), get_image_media_type(...)

Tests

pytest

The suite runs fully offline — no server, no API key, no network.

License

MIT