expectllm

Expect scripts for LLM conversations.

The insight: Agents are just expect scripts. Send a message, expect a pattern, branch on the match. That's it.

Table of Contents

• Why expectllm?
• Requirements
• Installation
• Quick Start
• Before/After
• Features
• API Reference
• Examples
• Environment Variables
• Prompting Tips
• Important Notes
• Contributing
• License

Why expectllm?

1. Zero boilerplate — No chains, no schemas, no output parsers. Just send and expect.
2. Pattern-first design — Use regex patterns you already know. The LLM adapts to your format, not the other way around.
3. Conversation as state machine — Each expect is a transition. Branch on match, retry on failure, build complex flows naturally.
4. Provider agnostic — Works with OpenAI, Anthropic, or any compatible API. Switch providers without changing code.
5. Debuggable — Every step is visible. No hidden prompts, no magic. Print the history, see what happened.
6. Lightweight — Single file, minimal dependencies. No framework lock-in.
7. Unix philosophy — Do one thing well. Compose with your existing tools.

Requirements

• Python 3.9+
• API key for at least one provider (OpenAI or Anthropic)

Installation

# Core only (no providers)
pip install expectllm

# With specific provider
pip install expectllm[openai]
pip install expectllm[anthropic]

# All providers
pip install expectllm[all]

Quick Start

from expectllm import Conversation

c = Conversation()
c.send("Is Python dynamically typed? Reply YES or NO")

if c.expect_yesno():
    print("Correct!")

That's it. Send a message, expect a pattern, branch on the result.

Before/After

Traditional approach (20+ lines):

from langchain.chat_models import ChatOpenAI
from langchain.output_parsers import PydanticOutputParser
from pydantic import BaseModel, Field
# ... setup, chains, error handling ...

expectllm (4 lines):

from expectllm import Conversation

c = Conversation()
c.send("Review this code. Reply with SEVERITY: low/medium/high",
       expect=r"SEVERITY: (low|medium|high)")
severity = c.match.group(1)

Features

Pattern-to-Prompt

When you pass an expect pattern to send(), expectllm automatically appends format instructions:

# You write:
c.send("Is this secure?", expect=r"^(YES|NO)$")

# expectllm sends:
# "Is this secure?
#
#  Reply with exactly 'YES' or 'NO'."

Expect Templates

No regex needed for common patterns:

# Extract JSON
c.send("Return user data as JSON")
data = c.expect_json()  # Returns dict

# Extract numbers
c.send("How many items?")
count = c.expect_number()  # Returns int

# Yes/No questions
c.send("Is this valid? Reply YES or NO")
if c.expect_yesno():  # Returns bool
    print("Valid!")

# Multiple choice
c.send("Classify as: bug, feature, or docs")
category = c.expect_choice(["bug", "feature", "docs"])

# Extract code
c.send("Write a Python function")
code = c.expect_code("python")  # Returns code string

API Reference

Conversation

c = Conversation(
    model="claude-sonnet-4-20250514",  # Optional, auto-detected from env
    system_prompt="You are helpful",   # Optional
    timeout=60,                        # Default timeout in seconds
    provider="anthropic",              # Optional: "openai" or "anthropic"
    max_history=20                     # Optional: limit conversation history
)

Methods

Properties

Exceptions

Examples

Extract Structured Data

from expectllm import Conversation

c = Conversation()
c.send("Parse this: 'Meeting with John at 3pm tomorrow'")
event = c.expect_json()  # {"person": "John", "time": "3pm", "date": "tomorrow"}

Multi-Turn Code Review

from expectllm import Conversation, ExpectError
import re

code = '''
def process_user(data):
    query = f"SELECT * FROM users WHERE id = {data['id']}"
    return db.execute(query)
'''

c = Conversation()
c.send(f"Review this code for security issues:\n```python\n{code}\n```")
c.expect(r"(found (\d+) issues|no issues found)", re.IGNORECASE)

if c.match.group(2) and int(c.match.group(2)) > 0:
    c.send("List the issues with severity ratings")
    c.expect(r"(critical|high|medium|low)", re.IGNORECASE)
    print(c.last_response)

Data Extraction

from expectllm import Conversation

text = "Contact John Smith at john@example.com or 555-1234"

c = Conversation()
c.send(f"""Extract contact info from:
{text}

Format:
NAME: <name>
EMAIL: <email>
PHONE: <phone>""")
c.expect(r"NAME: (.+)\nEMAIL: (.+)\nPHONE: (.+)")

print(f"Name: {c.match.group(1)}")
print(f"Email: {c.match.group(2)}")
print(f"Phone: {c.match.group(3)}")

Retry Pattern

from expectllm import Conversation, ExpectError

def analyze_document(text: str, max_retries: int = 3) -> dict:
    c = Conversation(system_prompt="You are a document analyzer.")
    c.send(f"Analyze this document and extract key entities:\n\n{text}")

    for attempt in range(max_retries):
        try:
            return c.expect_json()
        except ExpectError:
            if attempt < max_retries - 1:
                c.send("Please format your response as valid JSON.")

    raise ExpectError("Failed to extract JSON after retries")

Environment Variables

Set your API key:

# For Anthropic (Claude)
export ANTHROPIC_API_KEY="your-key"

# For OpenAI (GPT)
export OPENAI_API_KEY="your-key"

expectllm auto-detects the provider from the environment. Anthropic is preferred if both are set.

Prompting Tips

For reliable pattern matching:

1. Be explicit about format: "Reply with exactly 'YES' or 'NO'"
2. Use examples: "Format: SCORE: 8/10"
3. Constrain output: "Reply with just the number, nothing else"
4. Use code blocks: "Put your JSON in a ```json code block"

Important Notes

LLM Non-Determinism: LLM outputs are inherently non-deterministic. The same prompt may produce different responses across calls. For production use:

• Use explicit format instructions (expectllm does this automatically with expect=)
• Implement retry logic for critical extractions
• Consider temperature=0 for more consistent outputs
• Test patterns against varied response formats

Not a Testing Framework: expectllm is for scripting conversations, not for unit testing LLM outputs. For assertions about LLM behavior, combine with your existing test framework.

Contributing

See CONTRIBUTING.md for development setup and guidelines.

License

MIT License - see LICENSE for details.