SAIA

Framework-agnostic verb vocabulary for LLM agents

SAIA provides a fixed vocabulary of semantic verbs for LLM interactions. Instead of writing raw prompts, you express intent through verbs like ask, verify, critique, and refine.

Why a fixed vocabulary? The same insight that made SCUMM work for adventure games in 1987: constraints enable tooling. A finite set of operations means every interaction is debuggable, testable, and composable.

Why SAIA?

"Can't I just type this into Claude?" For a one-off question, yes. SAIA is for when you're building software, not chatting:

• Structured outputs - verify() returns VerifyResult(passed: bool, reason: str), not text you parse
• Composable - chain verify → critique → refine in 3 lines of code
• Testable - mock the backend, unit test your verbs
• Traceable - every verb call logged with inputs/outputs for debugging production issues
• Backend-agnostic - same code works with Anthropic, OpenAI, or local models
• Zero dependencies - pure Python core; bring your own LLM client

"Why not raw tool calling?" You could write the iteration loop yourself (~50-100 lines). SAIA gives you complete() with terminal detection, tracing, timeouts, and max iterations built in. It's requests vs urllib - both work, one is cleaner.

SAIA's value compounds when combining verbs, switching backends, or building a team around consistent patterns.

"Is this novel?" Perhaps not. These are the patterns that emerge when you build LLM agents that need to actually work. SAIA extracts them into ~2500 lines anyone can use, inspect, and build on.

Supported Python versions

CI tests every push against the full test suite:

• Linux (Ubuntu): CPython 3.11, 3.12, 3.13, 3.14

requires-python = ">=3.11" is declared in package metadata; newer Python versions are opt-in and validated against the full test suite before being added to the CI matrix.

Installation

pip install llm-saia               # zero-dep core
pip install 'llm-saia[pydantic]'     # + pydantic BaseModel schemas

The [pydantic] extra enables using pydantic.BaseModel as the schema type for complete_structured — see Custom typed output (Pydantic).

Quick Start

Verbs return a VerbResult — .value holds the typed result, .trace the per-step execution trace. Run the installed quick-start with python -m llm_saia.examples.quickstart (source at llm_saia/examples/quickstart.py); it uses an in-process canned backend so no API keys or network are needed.

from llm_saia import SAIA


async def main():
    saia = SAIA.builder().backend(your_backend).build()

    # Verify a claim
    result = await saia.verify(
        "def add(a, b): return a + b",
        "the function returns the sum of its two arguments",
    )
    print(f"Passed: {result.value.passed}, Reason: {result.value.reason}")

    # Generate counter-arguments
    critique = await saia.critique(
        "Adding more programmers to a late project always makes it finish faster."
    )
    print(f"Counter: {critique.value.counter_argument}")

    # Break down a complex task
    subtasks = await saia.decompose("Build a REST API with authentication")
    for task in subtasks.value:
        print(f"- {task}")

Verb Reference

Memory Verbs

Configuration

SAIA has no config files. Configuration is a Config dataclass constructed by the caller through the builder below — no YAML, no etc/ directory, no CLI, no file discovery.

Builder Pattern

saia = (
    SAIA.builder()
    .backend(backend)  # Required: LLM backend
    .tools(tool_defs, executor)  # Optional: tool calling
    .logger(logger)  # Optional: logging
    .system("You are a helpful assistant")  # Optional: system prompt
    .max_iterations(10)  # Optional: tool loop limit
    .max_call_tokens(4096)  # Optional: per-call token limit
    .build()
)

Runtime Modifiers

# Single LLM call (no tool loop)
result = await saia.with_single_call().verify(code, "compiles")

# Custom iteration limit
result = await saia.with_max_iterations(5).instruct(task)

# Timeout
result = await saia.with_timeout(30).decompose(problem)

# Correlation ID for tracing
result = await saia.with_request_id("req-123").ask(doc, question)

Output Guards

Guards validate LLM output and automatically retry with feedback if validation fails.

from llm_saia.guards import english_only, max_length, no_preamble

# Apply guards to any verb
result = await saia.with_guards(english_only(), no_preamble(), max_length(500)).ask(
    article, "summarize this"
)

Field-level guards for structured output:

from typing import Annotated
from llm_saia import Guarded
from llm_saia.guards import english_only, max_length


@dataclass
class Summary:
    title: Annotated[str, Guarded(english_only(), max_length(100))]
    body: Annotated[str, Guarded(english_only())]


result = await saia.extract(article, Summary)

Pre-built guards: english_only(), ascii_only(), max_length(n), no_emoji(), no_markdown(), no_preamble().

Iteration Guards

Iteration guards enforce behavioral constraints during a tool-calling loop, rather than validating the final result. When a guard fires, its feedback is injected into the conversation and the loop continues.

from llm_saia import IterationGuard


# Require the LLM to explain what it's doing before calling tools
def require_narrative(response):
    if response.tool_calls and not (response.content or "").strip():
        return "Explain what you're doing and why before calling tools."
    return None


result = await saia.with_guard(IterationGuard(require_narrative, name="narrative")).complete(task)

with_guard() and with_guards() accept both OutputGuard and IterationGuard — each is routed to the correct bucket automatically.

Tool Gates

Tool gates dynamically control which tools are visible to the model per iteration. Blocked tools are stripped from the outbound schema — the model never sees them and cannot spend output tokens generating a call that would be rejected.

from llm_saia import ToolGateContext

# Hide terminal tool until iteration 3 (shortcut via TerminalConfig)
saia = (
    SAIA.builder()
    .backend(backend)
    .tools(tools, executor)
    .terminal("complete_task", min_iterations=3)
    .build()
)


# Custom gate: only allow expensive tool after cheap tool has been called
def require_cheap_first(ctx: ToolGateContext) -> bool | str:
    if ctx.tool_call_counts.get("cheap_tool", 0) == 0:
        return "must call cheap_tool first"
    return True


result = await saia.with_tool_gate("expensive_tool", require_cheap_first).complete(task)

Gates return True/None to allow, False to block silently, or a str reason recorded in the trace (never surfaced to the model). See docs/tool-gates.md for advanced patterns including context factories for shared computation.

Custom typed output (Pydantic)

When no built-in verb fits — an LLM judge, a domain-specific extractor, a scoring rubric — reach for saia.complete_structured(prompt, schema). The prompt is sent verbatim (no framing added); the response is parsed into schema and returned as VerbResult[T].

schema can be a stdlib @dataclass (default; zero-dep) or a pydantic.BaseModel (requires pip install 'llm-saia[pydantic]'). With Pydantic, the full JSON-Schema vocabulary flows through — Field(ge=..., le=..., pattern=..., max_length=..., discriminator=..., ...) — and structural constraints are enforced by the backend's constrained decoder (vLLM, OpenAI). Custom @field_validator / @model_validator checks run on parse and can trigger SAIA's schema-retry loop.

from pydantic import BaseModel, Field
from typing import Literal


class EssayScore(BaseModel):
    score: float = Field(ge=0.0, le=10.0, description="0-10 overall grade")
    confidence: float = Field(ge=0.0, le=1.0)
    verdict: Literal["pass", "fail", "borderline"]
    feedback: str = Field(max_length=400, description="1-2 sentences")


result = await saia.complete_structured(
    f"Grade this essay for clarity and evidence:\n\n{essay}",
    EssayScore,
)

On backends with constrained decoding (vLLM, OpenAI strict mode), the decoder rejects tokens that would violate ge/le/pattern/etc. — the model literally cannot return score=85. Anthropic treats the schema as advisory; Pydantic validation and the schema-retry loop catch violations there. See docs/pydantic.md for the full story — enforcement matrix, validators + retry, discriminated unions, migration — and examples/pydantic_scorer.py for a runnable smoke test.

Examples

See the examples/ directory:

• investigate.py - Investigate a claim (verify → critique → refine)
• build.py - Build an app (decompose → instruct → synthesize)
• build_multi.py - Two LLMs collaborate (local generates, smart verifies)
• pydantic_scorer.py - Typed output with Pydantic (requires [pydantic] extra)

Design Philosophy

1. Verbs express intent - not implementation details
2. Structured over strings - type-safe dataclass responses
3. Composable primitives - build complex flows from simple verbs
4. Backend-agnostic - same code works with any LLM
5. Zero dependencies - pure Python core, you control your LLM client
6. Debuggable - every operation is traceable

Research Directions

SAIA's structured verb outputs create opportunities beyond inference:

• Consistency tuning - traces capture (prompt, decision) pairs that can fine-tune models for stable verb behavior. Same semantic question → same semantic answer.
• Structured generation - backends can use grammar-constrained decoding (xgrammar, outlines) to guarantee valid outputs from verbs like extract, verify, and classify without retry loops.

License

Apache 2.0 - see LICENSE

Maintained by LLM Works LLC and contributors.