tketool.llm

OpenAI-compatible model access, structured-output prompts, embeddings, and memory.

pip install tketool.llm

Chat Completions

import os

from tketool.llm import OpenAIChatModel

llm = OpenAIChatModel(
    model_name="gpt-4o-mini",
    apitoken=os.environ["OPENAI_API_KEY"],
    base_url="https://api.openai.com/v1",
    call_dict={"temperature": 0.2},
)

text = llm("用三句话解释向量检索", return_detail=False)
print(text)

Responses API

import os

from tketool.llm import OpenAIResponsesModel

llm = OpenAIResponsesModel(
    model_name="gpt-5-mini",
    apitoken=os.environ["OPENAI_API_KEY"],
    base_url="https://api.openai.com/v1",
    call_dict={"max_output_tokens": 300},
)

text, detail = llm("给出一个最小 RAG 流程", return_detail=True)
print(text)
print(detail)

Both transports accept a complete messages list and return the same detail shape. OpenAI-compatible gateways are supported by changing base_url and model_name.

messages = [
    {"role": "system", "content": "回答要简洁。"},
    {"role": "user", "content": "什么是结构化输出？"},
]

answer = llm("", return_detail=False, messages=messages)

Typed prompt input

PromptInvoker keeps the existing keyword-input API and also accepts a Pydantic model through invoke(model=None). In a prompt template, {.field} is a safe shorthand for a declared top-level model field:

from datetime import datetime

from pydantic import BaseModel
from tketool.llm import PromptInvokerPool


class GreetingInput(BaseModel):
    name: str
    time: datetime


pool = PromptInvokerPool(llm, lang="english", folder="prompts")
result = pool.invoke(
    "greeting",
    model=GreetingInput(name="Ada", time=datetime.now()),
)
print(result.result)

The greeting template can contain Hello {.name}; time {.time:%H:%M}. Internally, only model_dump(mode="python") values are passed to the template; the Python model object is not exposed for attribute traversal. Existing calls such as pool.get_invoker("greeting")(name="Ada", time="09:30") continue to work. If both forms provide the same field, the explicit keyword value wins.

The only public package path is tketool.llm; the retired tketool.lmc namespace and LMC-prefixed type aliases are not shipped.

Local Hugging Face embeddings are optional:

pip install "tketool.llm[local-embeddings]"

Memory

Memory is a small API backed by tketool.storage. Bind one instance to one user, agent, or project space, then use remember and recall:

from tketool.llm.memory import create_memory
from tketool.storage import MemoryBackend

storage = MemoryBackend()
memory = create_memory(storage=storage, space="users/user-001")

saved = memory.remember(
    "用户喜欢喝乌龙茶",
    kind="preference",
    tags=["profile", "drink"],
    metadata={"source": "chat"},
    idempotency_key="conversation-42/preference-1",
)

for item in memory.recall("用户喜欢喝什么？", limit=3):
    print(item.content, item.score)

memory.update(saved.id, tags=["profile", "confirmed"], if_revision=saved.revision)
memory.forget(saved.id)  # soft delete; pass hard=True for physical deletion
storage.close()

Choose persistence when constructing the storage backend; the memory API does not change:

import os

from tketool.storage import SQLiteBackend, create_backend

sqlite_storage = SQLiteBackend("memory.db")
postgres_storage = create_backend(os.environ["DATABASE_URL"])
# DATABASE_URL=postgresql+psycopg://user:password@localhost/app

Install the PostgreSQL driver with pip install "tketool.storage[postgresql]". MemoryBackend is process-local, SQLite is file-backed, and PostgreSQL uses the tketool.storage SQLAlchemy adapter.

Lexical retrieval is available by default. Semantic and entity channels are loaded only when their small protocols are injected:

memory = create_memory(
    storage=storage,
    space="users/user-001",
    embedder=my_embedder,                 # implements embed(text) -> list[float]
    entity_extractor=my_entity_extractor, # implements extract(text) -> Iterable[str]
)

semantic = memory.recall("饮品偏好", using=["semantic"])
hybrid = memory.recall("Alice 的偏好", using=["lexical", "semantic", "entity"])
memory.reindex(using=["semantic"])  # after changing the embedding model/version

The built-in OpenAI-compatible embedding provider and tokenizer implement those protocols directly:

import os

from tketool.llm import OpenAIEmbeddingProvider
from tketool.llm.memory import SimpleTokenizer, create_memory
from tketool.storage import MemoryBackend

storage = MemoryBackend()
memory = create_memory(
    storage=storage,
    tokenizer=SimpleTokenizer(),
    embedder=OpenAIEmbeddingProvider(
        model_name="text-embedding-3-small",
        apitoken=os.environ["OPENAI_API_KEY"],
        base_url="https://api.openai.com/v1",
    ),
)

memory.remember("用户喜欢喝乌龙茶")
print(memory.recall("饮品偏好", using=["semantic"]))

For an offline local model, use LocalTransformerEmbeddingProvider with local_files_only=True. It resolves a cached Hugging Face snapshot without a network probe. Entity extraction remains application-specific: inject any object implementing extract(text) -> Iterable[str].

See tketool.llm.memory.examples for runnable memory, SQLite, and PostgreSQL examples. The legacy agent/context implementation was removed because it depended on the retired scheduler. Agent runtime code now lives under tools/agent_framework and is not part of the tketool.llm distribution.

External agent control

from tketool.llm.agent_proxy import create_agent_proxy, SSHConnection controls Codex app-server or Hermes ACP with async sessions, streamed tool/text events, interactions and cancellation. Local and SSH connections use the same API; remote tools operate on remote files. Hermes client schema support is optional: pip install "tketool.llm[hermes]". Agents must already be installed and authenticated.

This is a control client, separate from the retired scheduler-based agent runtime. It does not replay failed agent tasks or guarantee execution survives disconnection. See agent proxy architecture and examples.