📦 llm-sdk

Small Python SDK for OpenAI-compatible LLM APIs.

One file, clean API, boring on purpose. Use it with local servers, OpenAI-style endpoints, structured output, tool calls, vision inputs, and reasoning streams.

✨ Features

• Sync and async clients
• Streaming and non-streaming responses
• Configurable retry with per-call override
• OpenAI Chat Completions support
• Optional OpenAI Responses API mode
• Structured output from JSON schema or typed Python classes
• Tool schema generation from Python callables
• Vision input normalization from URL, path, base64, or PIL image
• Thinking/reasoning token parsing
• Lightweight verbose stats for streams

🚀 Get Started

Install the package directly from PyPI:

pip install llm-sdk-py

If you also need PIL image support:

pip install "llm-sdk-py[pillow]"

Alternatively, since it's designed to be simple, you can still just drop llm_sdk.py directly into your project!

from llm_sdk import LLM

llm = LLM(
    model="qwen3.6-27b",
    base_url="http://localhost:1234",
    api_key="lm-studio",
)

response = llm.response(system="You're a helpful assistant!",input="Write a tiny haiku about fast code.")

print(response["answer"])

By default, base_url="http://localhost:1234/v1" and api_key="lm-studio", so local LM Studio-style servers work with very little setup.

All inference methods accept either input="..." for the common single-user-message case or a Chat Completions-style message list + system for a system prompt:

response = llm.response(messages=[
    {"role": "user", "content": "Write a tiny haiku about fast code."},
])

📡 Streaming

for event in llm.stream_response(input="Explain adapters in one paragraph."):
    if event["type"] == "answer":
        print(event["content"], end="", flush=True)

Events are small dictionaries:

{"type": "answer", "content": "..."}
{"type": "reasoning", "content": "..."}
{"type": "tool_call", "content": {"id": "...", "name": "...", "arguments": {...}, "callable": Callable}}
{"type": "tool_call_part", "content": {"id": "...", "name": "...", "args_delta": "{\"city\": \"Berlin\"}"}}
{"type": "verbose", "content": {"tokens": 42, "tokens_per_second": 91.3, "latency": 0.2, "prompt_tokens": 10, "completion_tokens": 32, "total_tokens": 42, "stop_reason": "stop"}}
{"type": "final", "content": {"answer": "...", "stop_reason": "stop"}}
{"type": "done"}

Use final=True if you also want a final aggregated response event.

⏱️ Async

import asyncio
from llm_sdk import LLM

async def main():
    async with LLM(model="gpt-5.5", api_key="sk-...", base_url="https://api.openai.com/v1", use_responses_api=True) as llm:
        response = await llm.async_response(input="Give me a crisp project name.")
        print(response["answer"])

asyncio.run(main())

🔄 Retry

Set max_retries globally or override it per call.

# global
llm = LLM(model="qwen3.6-27b", max_retries=5)
# per call
llm.response(input="...", max_retries=0)

📐 Structured Output

Pass a JSON schema or a typed class. Classes are converted into OpenAI-compatible JSON schema (including enums, literals, and optional fields).

class Verdict:
    sentiment: str
    score: float
    tags: list[str]

result = llm.response(
    input="Review: fast, small, surprisingly nice.",
    output_format=Verdict,
)

print(result["answer"])

🛠️ Tools

Pass Python callables or already-built OpenAI tool definitions. The SDK exposes tool definitions and returns streamed/final tool calls.

It does not execute tools for you. You stay in control.

def search_docs(query: str, limit: int = 5) -> str:
    """Search internal docs."""
    return "..."

response = llm.response(
    input="Find the auth setup notes.",
    tools=[search_docs],
)

print(response.get("tool_calls", []))

For multi-turn conversation loops with tool calls, use the built-in helper functions to easily format messages:

from llm_sdk import assistant_message, system_message, tool_result, user_message

# 0. Optional system message
msg0 = system_message("You are a helpful assistant.")

# 1. Format the assistant's response (includes both answer text and tool calls)
msg1 = assistant_message(response)

# 2. Format a tool call execution result
msg2 = tool_result(response["tool_calls"][0], "result string or dict")

# 3. Format subsequent user messages
msg3 = user_message("Tell me more about the results.")

👁️ Vision

Image content can be a URL, a local path, base64, or a PIL image.

response = llm.response([
    {
        "role": "user",
        "content": [
            {"type": "text", "text": "What is in this image?"},
            {"type": "image", "image_path": "photo.png"},
        ],
    }
])

Supported image forms include:

• {"type": "image", "image_url": "https://..."}
• {"type": "image", "image_path": "local-file.png"}
• {"type": "image", "image_base64": "..."} (MIME type auto-detected from the bytes)
• {"type": "image", "image_pil": image}

🔌 Responses API

Use use_responses_api=True for endpoints that prefer OpenAI's Responses API shape.

llm = LLM(
    model="gpt-5.5",
    api_key="sk-...",
    base_url="https://api.openai.com/v1",
    use_responses_api=True,
)

🧠 Reasoning Effort

Use reasoning_effort="high" to set the model's reasoning effort.

response = llm.response(
    input="...",
    reasoning_effort="high"
)

⚙️ API

• response(...) and stream_response(...)
• async_response(...) and async_stream_response(...)
• input="..." or messages=[...] for all inference methods
• LLM.list_models(...) / LLM.async_list_models(...) and standalone list_models(...) / async_list_models(...)
• max_retries=3 globally on LLM(...) or per call
• reasoning_effort="low|medium|high" where supported
• hide_thinking=False to stream/return reasoning content
• CustomThinkingToken(...) for custom <think>-style parsing
• verbose=True for token-ish stream stats (incl. stop_reason)
• stop_reason in final responses: "stop" | "length" | "tool_calls" | "content_filter" (and "failed"/"cancelled"/"incomplete" from the Responses API)
• message helpers: system_message(...), user_message(...), assistant_message(...), tool_result(...)
• with LLM(...) as llm: / async with LLM(...) as llm: for cleanup
• from llm_sdk import __version__ for the package version

💡 Why

Most LLM wrappers either become frameworks or stay too close to raw HTTP. This sits in the middle: enough structure to be pleasant, little enough surface area to understand in one sitting.

If this saves you time, please ⭐ the repo! Thanks! ♥️

📜 License

MIT