llmutil

This library provides tools to generate structured output and function calling from the OpenAI API.

What is Structured Output?

Structured Output is the recommended method for getting formatted responses. It guarantees that outputs follow your defined schema, making it more reliable than previous methods like JSON mode, function calls, or tool use.

How to Use

Structured Output

To use Structured Output, you need to define a schema using a simple dictionary format:

from llmutil import new_response

output = new_response(
    [
        {
            "role": "user",
            "content": "normalize this address: 1 hacker way, menlo park, california",
        }
    ],
    model="gpt-4.1-mini",
    schema={
        "street": "string",
        "city": "string",
        "state": "string",
    },
)
# {'street': '1 Hacker Way', 'city': 'Menlo Park', 'state': 'CA'}

Function Calling

For function calling, implement the Tooling protocol to define available functions:

from llmutil import Result, Tooling, new_response

def add(a, b):
    return a + b

class UseAdd(Tooling):
    def on_function_call(self, name, args):
        if name == "add":
            return Result(add(args["a"], args["b"]))

    def get_tools(self):
        return {
            "add": {
                "a": "number",
                "b": "number",
            }
        }

messages = [
    {
        "role": "system",
        "content": "you cannot do math. you must use the add() function to add numbers.",
    },
    {
        "role": "user",
        "content": "alice has 10 apples, bob has 20 apples, how many apples do they have in total?",
    },
]

output = new_response(messages, model="gpt-4.1-mini", tooling=UseAdd())
# Alice and Bob have 30 apples in total.

API Reference

new_response(messages, *, model, tooling=None, schema=None, memory=None, timeout=30)

Main function for generating responses from OpenAI API.

• messages: List of message dictionaries in OpenAI format
• model: OpenAI model name (e.g., "gpt-4.1-mini")
• tooling: Optional Tooling implementation for function calling
• schema: Optional dictionary defining the output schema
• memory: Optional vector store ID for file search
• timeout: Request timeout in seconds (default: 30)

Tooling Protocol

Protocol for implementing function calling:

• on_function_call(self, name: str, args: dict): Handle function calls, return Result to continue or any other value to return immediately
• get_tools(self): Return dictionary mapping function names to parameter schemas

Result Class

Wrapper for function call results that should continue the conversation:

• Result(result): Wrap a result to continue the conversation flow