urai

Urai (உரை) means “dialogue” in Tamil. It is the part of a chat program that does not depend on which model you use. Local engines such as llama.cpp and hosted APIs such as OpenAI’s all take a list of messages and return a reply, but each one names its settings differently, formats tool calls differently, and reports token usage differently. A program that wants to switch between them, or move one conversation from one to another, ends up rewriting the same glue for each.

Urai writes that glue once. It defines one conversation format, one set of option names, one tool loop, one streaming format and one way of counting tokens. A backend package such as rishi supplies the other half: a Chat subclass that knows how to send one request to its engine. aidialog defines the typed message parts that some backends use.

Install

pip install uraiyadal

Urai contains no backend, so on its own it cannot talk to a model. Install a backend package as well and import it; importing it registers its runtimes with urai. The examples below assume a backend has registered a runtime called remote for hosted models, as rishi does.

Start a chat

mk_chat builds a chat for a model name, and calling the chat with a message runs one turn and returns the reply. resp_text gets the text out of the reply.

from urai import mk_chat, resp_text

chat = mk_chat('gpt-5.1', runtime='remote', sp='Answer briefly.', temp=0.2)
reply = chat('What is 6 times 7?')
print(resp_text(reply))

The conversation so far is in chat.hist, as a list of plain dicts in the OpenAI chat format. Every backend reads and writes this same format, so a history from one backend can be passed to another with messages= and the conversation continues there.

[(msg['role'], resp_text(msg)) for msg in chat.hist]

Resolve a model before loading it

Loading a local model is slow, and you may want to know what you are about to load first. resolve works out which runtime a model name belongs to and how large its context window is, without building anything:

from urai import resolve

spec = resolve('claude-sonnet-4.6', runtime='remote')
print(spec.runtime, spec.ctx, spec.local)

chat = mk_chat(spec)

The result is a ModelSpec. It is frozen, so it can be cached, compared, and passed from one part of a program to another, and mk_chat accepts it directly.

Configure one turn

Options passed when the chat is built apply to the whole conversation. Options passed to a single call apply only to that turn:

reply = chat('Explain the hard part.', effort='high', temp=0.9)

Option names are the same for every backend: ctx for the context window, temp for temperature, max_output_tokens, think, effort and so on. Each backend translates them into its engine’s own names and silently drops the ones its engine cannot use. A keyword urai does not know, such as llama.cpp’s n_gpu_layers, is passed through to the backend unchanged.

Add a tool

A tool is an ordinary Python function. Urai builds the schema the model sees from the function’s name, type annotations and docstring:

def add(a: int, b: int) -> int:
    "Add two integers."
    return a + b

chat = mk_chat('gpt-5.1', runtime='remote', tools=[add], approve=lambda call: True)
reply = chat('Use the tool to add 20 and 22.')
print(resp_text(reply))

When the model asks for a tool, urai calls approve with the call; if it returns True the function runs and its result goes back to the model, which then continues. This repeats until the model answers without calling a tool, or until max_steps calls (10 by default) have been made in the turn. Leaving out approve runs every call. A tool the provider runs on its own servers, such as a hosted web search, is recorded in the history but never run locally.

Modules

Write a backend

A backend is a subclass of Chat registered under a runtime name. Add ToolLoopMixin when the engine returns tool calls as data, and implement the two methods it needs: _model_step, which makes one request and returns the reply, and _stream_step, which does the same while yielding chunks. _opt_map renames portable options to the engine’s own, and _opt_skip lists options the engine cannot use.

from urai import Chat, ToolLoopMixin, Runtime, register_runtime

class MyChat(ToolLoopMixin, Chat):
    _runtime = 'mine'
    _opt_map = {'ctx': 'n_ctx'}
    _opt_skip = ('effort',)

    def _model_step(self, **kw): ...
    def _stream_step(self, **kw): ...

register_runtime(Runtime('mine', 'mypkg.chat.MyChat', pats=('.mine',)))

After this, Chat('model.mine') returns a MyChat, because the name contains .mine.

Develop

The notebooks in nbs/ are the source; the files in urai/ are generated from them.

uv sync --all-extras --group dev
uv run nbdev-export
uv run nbdev-test
uv run nbdev-clean

The tests use scripted backends, so they load no model and need no network.