wiltech-labs-ai

AI building blocks for Wiltech FastAPI apps:

• Providers: Workers AI (binding or REST), Groq and Ollama/any OpenAI-compatible server behind one AI protocol.
• Tool calling: OpenAICompatibleLlm over plain httpx (no openai package, which costs about 1.4 s of CPU on a Python Worker cold start).
• Embeddings: Workers AI or Ollama.
• Vector stores: one VectorStore protocol with three adapters: the app's own database, Cloudflare Vectorize, Qdrant.
• Hybrid search: reciprocal_rank_fusion to merge vector and keyword rankings.
• Agent loop: ToolAgent runs tools as the server-supplied user and returns the answer with a trace.

Builds on wiltech-labs-core (get_config, Database, errors). Pure Python (httpx), so it runs in Cloudflare Workers.

Install

uv add wiltech-labs-ai

Usage

Plain chat

from wiltech_labs_ai import get_ai

ai = get_ai(request, provider)              # "cloudflare" | "groq" | "local"
reply = await ai.run(model, {"messages": [{"role": "user", "content": "Hello"}]})
reply["response"]

"Ask your data": tools + agent

from wiltech_labs_ai import TOOL_RESULTS_ARE_DATA, Tool, ToolAgent, get_tool_llm


async def count_todos(user_id: str, args: dict) -> dict:
    # user_id comes from the server (require_owner), never from the model
    return {"count": await todo_repository.count(user_id, state=args.get("state"))}


tools = [Tool("count_todos", "Count the user's todos, optionally by state.",
              {"type": "object", "properties": {"state": {"type": "string", "enum": ["NEW", "DONE"]}}},
              count_todos)]

llm, model = get_tool_llm(request, provider)      # 503 NotConfiguredError if a key is missing
result = await ToolAgent(llm, model, tools).run(user_id, [
    {"role": "system", "content": f"Answer questions about the user's data. {TOOL_RESULTS_ARE_DATA}"},
    {"role": "user", "content": question},
])
result.answer, result.tool_calls, result.completed

Embeddings + vector store + hybrid search

from wiltech_labs_ai import DatabaseVectorStore, VectorRecord, get_embedder, reciprocal_rank_fusion
from wiltech_labs_core import get_database

embed, model = get_embedder(request)
store = DatabaseVectorStore(get_database(request), namespace="todo")

# write path
[vector] = await embed([f"{todo.title}\n{todo.description or ''}"])
await store.upsert([VectorRecord(todo.id, scope=todo.user_id, vector=vector, model=model)])

# query path: meaning + exact words, fused
[query_vector] = await embed([q])
vector_hits = await store.query([user_id], query_vector, limit=30)
keyword_hits = await todo_repository.search_by_keyword(user_id, q, limit=30)
ranked = reciprocal_rank_fusion([m.id for m in vector_hits], [t.id for t in keyword_hits])

# backfill: embed only what's missing
missing = set(all_ids) - await store.existing_ids(all_ids)

scope is the tenancy boundary: the owner's user id for private data, or a group id for group data (query with every group the caller can see). Every query filters by it.

Choosing a store

All three are interchangeable behind VectorStore. Switching store means re-embedding (run your backfill against the new store).

Configuration

Public API

Development

uv run --package wiltech-labs-ai pytest packages/ai

Publishing

Publish wiltech-labs-core first; this package depends on it.

cd packages/ai

# 1. Test, bump, build
uv run pytest
uv version --bump patch                 # or minor / major; skip for the first 0.1.0
rm -rf ../../dist && uv build           # -> ../../dist/

# 2. TestPyPI (optional rehearsal), using a token from test.pypi.org
read -rsp "TestPyPI token: " UV_PUBLISH_TOKEN && export UV_PUBLISH_TOKEN && echo
uv publish --index testpypi ../../dist/*
unset UV_PUBLISH_TOKEN

# 3. PyPI (the real thing), using a token from pypi.org
read -rsp "PyPI token: " UV_PUBLISH_TOKEN && export UV_PUBLISH_TOKEN && echo
uv publish ../../dist/*
unset UV_PUBLISH_TOKEN

# 4. Commit and tag
git commit -am "wiltech-labs-ai <version>"
git tag ai-v<version> && git push && git push --tags

A version number can only be uploaded once to each site. To retry after a fix, bump the version and rebuild.