llm-ai-api

A thin, stable HTTP wrapper for the OpenAI, OpenRouter, Claude (Anthropic), and Google APIs. It does not use the official SDKs; the only dependency is httpx.

日本語版 (Japanese)

Installation

pip install llm-ai-api

Quick start

from llm_ai_api import llm

luna = llm("openai:gpt-6-luna")
print(luna.ask("Come up with a cute name for a dog"))

Set the API key via an environment variable (OPENAI_API_KEY, OPENROUTER_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY or GOOGLE_API_KEY). The model is always "<provider>:<model>" (providers: openai, openrouter, anthropic / claude, google). Google Gemini is called through Google's OpenAI-compatible endpoint.

Streaming and chat

# Streaming
for piece in luna.stream("Write a short story"):
    print(piece, end="", flush=True)

# Using a chat history
result = luna.chat([{"role": "user", "content": "hi"}])  # ChatResult: text, stop_reason, usage, raw

Parameters you do not pass are never sent (newer models reject even default values such as temperature). Anything else can go through extra={...}, or the raw layer: luna.client.request(path, body).

You can also pick from the model constants (they are just "provider:model" strings):

from llm_ai_api import llm, models

m = llm(models.ANTHROPIC_CLAUDE_OPUS_5_5)

The constants are examples. Any other model works as a plain string, e.g. llm("openai:some-new-model").

Async

The same llm() works with asyncio: add an a to the method name (aask(), achat(), astream()).

import asyncio
from llm_ai_api import llm

async def main():
    async with llm("openai:gpt-6-luna") as m:
        print(await m.aask("Come up with a cute name for a dog"))

        async for piece in m.astream("Write a short story"):  # no await here
            print(piece, end="", flush=True)

        # One LLM can be shared by concurrent calls
        answers = await asyncio.gather(m.aask("One dog name"), m.aask("One cat name"))
        print(answers)

asyncio.run(main())

• astream() starts the request at the first iteration, so connection and HTTP errors are raised there (when async for starts), not when astream() is called (the sync stream() raises on the call). To stop early, call await stream.aclose().
• Close it with async with (or await m.aclose()). A closed LLM cannot be reused.
• Do not carry one across event loops: create a new llm() for each asyncio.run().

The cache and the log work the same way as in the sync API. Only the common interface (llm()) is async; there are no async provider clients or raw layer (request()).

To try things step by step from the beginning, see the Tutorial.

Response caching

Off by default. When on, a repeat call with the same provider, model and request (messages and parameters) returns the stored result without calling the API.

from llm_ai_api import llm, set_cache

set_cache("cache.db")                    # process-wide: on (an SQLite file path)
m = llm("openai:gpt-6-luna")              # uses the global setting
m2 = llm("openai:gpt-6-luna", cache="other.db")  # per instance
m3 = llm("openai:gpt-6-luna", cache=False)       # off for this instance only

cache=True uses the default file llm_ai_api_cache.db. A SQLiteCache object can also be passed. Setting the environment variable LLM_AI_API_CACHE_DIR to a directory turns the cache on as <dir>/cache.db without calling set_cache(). Streaming (stream() / stream_chat()) is cached too (stored only when read to the end).

Stored responses are encrypted with a key derived from the API key. The lookup key is a hash of {provider}:{model}@{api_key}>>>{input}, so a different API key means a separate cache. Remove old entries with clear_cache(days) (clear_cache(0) deletes everything).

Note that while the cache is on, the same request always returns the same stored answer. Clear the cache or turn it off when you want a fresh generation. When the library creates the file, it is owner-only (0600).

See docs/io_cache.md for details.

Input/output log

Off by default. When enabled, every ask() / chat() / stream() / stream_chat() call appends one JSON Lines record with the input (request body) and the output (raw response).

from llm_ai_api import llm, set_log

set_log("llm_log.jsonl")                       # on for the whole process
m = llm("openai:gpt-6-luna", log="other.jsonl")  # per instance
m2 = llm("openai:gpt-6-luna", log=False)         # off for this instance only

Set the LLM_AI_API_LOG_DIR environment variable to a directory to enable it as <dir>/llm.jsonl. Each line has time, provider, model, stream, cached, request and response. The API key is never recorded. The log is plain text and contains your prompts and the answers, so protect it. A stream is recorded only after it has been read to the end.

See docs/io_logs.md for details.

Examples

See examples/. Each script takes a provider:model argument (or the LLM_MODEL env var).

Script	What it shows
hello.py	Ask for a greeting
hello_stream.py	Print the reply as it streams
hello_async.py	aask() and astream() with asyncio.run and async with
chat_history.py	Reply based on a chat history
ocr_test.py	Read the text in an image
image_desc.py	Describe an image
models_test.py	Two models (GPT-6 Luna and Claude Haiku) discuss the breakfast menu, using the models constants (needs both API keys)
cache_demo.py	Ask the same question twice and see the second answer come from the cache

Image content is not converted between providers; the examples build the right format for each one (see examples/_common.py).

Error messages

Error messages are available in English and Japanese. The language is chosen from LLM_AI_API_LANG, then LANGUAGE / LC_ALL / LC_MESSAGES / LANG, and falls back to English. You can also set it explicitly:

from llm_ai_api import llm, set_language

set_language("ja")                       # for the whole process
m = llm("openai:gpt-6-luna", lang="en")  # or per model

Every exception has a language-independent code (e.g. "missing_api_key") you can check instead of the message. Messages returned by the API itself are never translated.

Prompt caching

See docs/cache.md.

License

MIT