thinllm 0.1.6a1

Add your description here

ThinLLM

A thin, unified wrapper for LLM interactions with support for multiple providers (OpenAI, Azure OpenAI, Anthropic, AWS Bedrock, and Gemini).

⚠️ Under Development: This project is currently under active development. APIs may change, and some features may be incomplete or subject to modification.

Table of Contents

• Features
• Why ThinLLM?
• Key Concepts
• Installation
• Quick Start
  • Basic Usage
  • Streaming Responses
  • Structured Output
  • Function Calling
• Providers
  • OpenAI
  • Azure OpenAI
  • Anthropic
  • AWS Bedrock
  • Google Gemini
• API Reference
• Examples
• Troubleshooting
• Roadmap
• Contributing
• License

Features

• Single Function API: One llm() function for all providers - no need to learn multiple APIs
• Provider Support: OpenAI, Azure OpenAI, Anthropic (Claude), AWS Bedrock, and Google Gemini
• Streaming: Full support for streaming responses
• Structured Output: Get Pydantic models directly from LLMs with type safety
• Function Calling: Tool/function calling with automatic serialization
• Type Safety: Full type hints and runtime validation with Pydantic

Why ThinLLM?

One function. All providers. Zero hassle.

Building applications with multiple LLM providers means learning different APIs, handling different response formats, and managing provider-specific quirks. ThinLLM eliminates this complexity:

• Single llm() Function: One function for all your LLM needs - no need to learn provider-specific APIs
• Write Once, Use Anywhere: Same code works with OpenAI, Claude, Bedrock, and Gemini
• Provider Agnostic: Switch providers by changing just the config - no code refactoring needed
• Type Safety: Full Pydantic integration for validated, structured outputs
• Minimal Overhead: Thin wrapper that stays close to native provider APIs
• Production Ready: Battle-tested with comprehensive test coverage

# Switch providers by updating the config
config = LLMConfig(
    provider="anthropic",      # Was "openai"
    model_id="claude-sonnet-4", # Was "gpt-4"
    params=ModelParams(temperature=0.7)
)
# Same llm() function, same messages, same code structure!
response = llm(config, messages)

Key Concepts

ThinLLM is built around a single powerful principle: one function for all your LLM needs.

Instead of learning different APIs for OpenAI, Anthropic, Bedrock, and Gemini, you just use the llm() function. Change providers by switching one configuration parameter - everything else stays the same.

Core Components

1. llm() function: The only function you need - handles all LLM interactions
2. LLMConfig: Configure which provider and model to use
3. ModelParams: Standard parameters (temperature, max_tokens, etc.) that work across all providers
4. Messages: Use SystemMessage, UserMessage, and AIMessage to build conversations
5. Structured Output: Pass a Pydantic model as output_schema to get validated, typed responses
6. Tools: Pass Python functions as tools parameter for function calling

Installation

Basic Installation

pip install thinllm

Provider-Specific Dependencies

Install dependencies for the providers you want to use:

# For OpenAI
pip install thinllm[openai]

# For Azure OpenAI
pip install thinllm[azure-openai]

# For Anthropic (Claude)
pip install thinllm[anthropic]

# For AWS Bedrock with Anthropic models
pip install thinllm[bedrock]

# For Google Gemini
pip install thinllm[gemini]

# For all providers
pip install thinllm[all]

Environment Setup

Set your API keys as environment variables or use a .env file:

export OPENAI_API_KEY=your-openai-key-here
export ANTHROPIC_API_KEY=your-anthropic-key-here
export GEMINI_API_KEY=your-gemini-key-here

# Azure OpenAI credentials
export AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com
export AZURE_OPENAI_API_KEY=your-azure-api-key  # Optional if using Entra ID

# AWS credentials (for Bedrock)
export AWS_ACCESS_KEY_ID=your-aws-access-key
export AWS_SECRET_ACCESS_KEY=your-aws-secret-key
export AWS_REGION=us-east-1

Or create a .env file in your project:

OPENAI_API_KEY=your-openai-key-here
ANTHROPIC_API_KEY=your-anthropic-key-here
GEMINI_API_KEY=your-gemini-key-here
AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com
AZURE_OPENAI_API_KEY=your-azure-api-key
AWS_ACCESS_KEY_ID=your-aws-access-key
AWS_SECRET_ACCESS_KEY=your-aws-secret-key
AWS_REGION=us-east-1

Quick Start

Basic Usage

Here's how to make your first LLM call:

from thinllm import llm, LLMConfig, ModelParams, UserMessage
from dotenv import load_dotenv

# Load your API keys
load_dotenv()

# Configure your LLM
config = LLMConfig(
    provider="openai",  # or "azure_openai", "anthropic", "bedrock_anthropic", "gemini"
    model_id="gpt-4",
    params=ModelParams(temperature=0.7, max_output_tokens=1024)
)

# Ask a question
messages = [UserMessage(content="What is the capital of France?")]
response = llm(config, messages)

# Print the response
print(response.content[0].text)
# Output: The capital of France is Paris.

Streaming Responses

Stream responses in real-time for better user experience:

# Stream the response - each chunk contains the complete response up to that point
for chunk in llm(config, messages, stream=True):
    if chunk.content:
        print(chunk.content[0].text)
        # Note: chunk.content[0].text contains the full text generated so far, not just the diff

Structured Output with Pydantic

Get validated, structured data directly from the LLM:

from pydantic import BaseModel
from thinllm import SystemMessage

class CalendarEvent(BaseModel):
    thought: str | None = None
    name: str | None = None
    date: str | None = None
    participants: list[str] | None = None

messages = [
    SystemMessage(content="Extract the event information."),
    UserMessage(content="Alice and Bob are going to a science fair on Friday."),
]

# Get structured output that matches your Pydantic model
response = llm(config, messages, output_schema=CalendarEvent)

print(f"Event: {response.name}")
print(f"Date: {response.date}")
print(f"Participants: {', '.join(response.participants)}")

Function Calling / Tools

Enable the LLM to request function calls:

from thinllm import SystemMessage

# Define a simple function
def get_horoscope(sign: str):
    """Get the horoscope for a zodiac sign."""
    return f"{sign}: Next Tuesday you will befriend a baby otter."

messages = [
    SystemMessage(content="You are a helpful assistant."),
    UserMessage(content="What is my horoscope? I am an Aquarius."),
]

# The LLM will indicate which function to call
response = llm(config, messages, tools=[get_horoscope])

# Check if the LLM requested a tool call
if response.get_tool_call_contents():
    tool_call = response.get_tool_call_contents()[0]
    print(f"LLM requested function: {tool_call.name}")
    print(f"With arguments: {tool_call.input}")
    
    # Execute the tool and continue the conversation
    messages.append(response)
    messages.append(
        UserMessage(content=[tool_call.get_tool_result(tools=[get_horoscope])])
    )
    
    # Get the final response after tool execution
    final_response = llm(config, messages, tools=[get_horoscope])
    print(final_response.content[0].text)

Providers

OpenAI

from thinllm import LLMConfig, ModelParams

config = LLMConfig(
    provider="openai",
    model_id="gpt-4",
    params=ModelParams(
        temperature=0.7,
        max_output_tokens=4096
    )
)

Supported Models: gpt-4, gpt-4-turbo, gpt-3.5-turbo, gpt-4o, etc.

Setup: Set OPENAI_API_KEY environment variable

Azure OpenAI

Use OpenAI models through Azure's enterprise platform with enhanced security and compliance:

from thinllm import Credentials

# With API Key authentication
config = LLMConfig(
    provider="azure_openai",
    model_id="gpt-4o",  # Your deployment name in Azure
    params=ModelParams(
        temperature=0.7,
        max_output_tokens=4096
    ),
    credentials=Credentials(
        azure_endpoint="https://your-resource.openai.azure.com",
        api_key="your-azure-api-key"
    )
)

# With Microsoft Entra ID (Azure AD) authentication
config = LLMConfig(
    provider="azure_openai",
    model_id="gpt-4o",  # Your deployment name in Azure
    params=ModelParams(
        temperature=0.7,
        max_output_tokens=4096
    ),
    credentials=Credentials(
        azure_endpoint="https://your-resource.openai.azure.com",
        # No api_key - will use DefaultAzureCredential
    )
)

Supported Models: Any OpenAI model deployed in your Azure OpenAI resource

• gpt-4o, gpt-4o-mini, gpt-4-turbo, gpt-4, gpt-3.5-turbo
• Note: Use your Azure deployment name as model_id

Setup:

1. Create an Azure OpenAI resource in Azure Portal
2. Deploy a model and note the deployment name
3. Get your endpoint URL from the resource
4. Choose authentication:
   • API Key: Get from "Keys and Endpoint" section in Azure Portal
   • Microsoft Entra ID: Run az login or configure managed identity

Environment Variables:

export AZURE_OPENAI_ENDPOINT="https://your-resource.openai.azure.com"
export AZURE_OPENAI_API_KEY="your-api-key"  # Optional if using Entra ID

Authentication Options:

• API Key: Simple, good for development and testing
• Microsoft Entra ID: Enterprise-grade, recommended for production
  • Uses DefaultAzureCredential from azure-identity
  • Supports: Azure CLI, managed identities, environment variables, and more

Anthropic (Claude)

config = LLMConfig(
    provider="anthropic",
    model_id="claude-sonnet-4",
    params=ModelParams(
        temperature=0.7,
        max_output_tokens=4096
    )
)

Supported Models: claude-sonnet-4, claude-opus-4, claude-3-5-sonnet-20241022, etc.

Setup: Set ANTHROPIC_API_KEY environment variable

Anthropic-Specific Features

Prompt Caching (Reduce costs and latency):

Anthropic's prompt caching allows you to cache frequently used content blocks (like system prompts, large documents, or context) to reduce costs and improve response times.

from thinllm import (
    llm, LLMConfig, SystemMessage, UserMessage,
    InputTextBlock, ContentExtra, AnthropicCacheControl
)

# System prompt with caching
system_msg = SystemMessage(content=[
    InputTextBlock(text="You are a helpful assistant."),
    InputTextBlock(
        text="Here is a large document to reference: ...",
        extra=ContentExtra(
            # Cache with default TTL (5 minutes)
            anthropic_cache_control=AnthropicCacheControl()
        )
    )
])

# User message with cached context
user_msg = UserMessage(content=[
    InputTextBlock(
        text="Long context that will be reused across multiple requests...",
        extra=ContentExtra(
            # Cache for 1 hour
            anthropic_cache_control=AnthropicCacheControl(ttl="1h")
        )
    ),
    InputTextBlock(text="What is the main theme?")
])

config = LLMConfig(
    provider="anthropic",
    model_id="claude-sonnet-4",
    params=ModelParams(max_output_tokens=1024)
)

response = llm(config, [system_msg, user_msg])

Cache Control Options:

• ttl: Time-to-live for cache - "5m" (5 minutes, default) or "1h" (1 hour)
• enabled: Set to False to explicitly disable caching (default: True)
• Omit ttl to use Anthropic's default TTL (5 minutes)

Best Practices:

• Cache large, reusable context (system prompts, documents, examples)
• Use longer TTL ("1h") for stable content
• Cache control is ignored by other providers (Anthropic-only feature)
• Not supported for reasoning/thinking content blocks

Extended Thinking with Interleaved Thinking:

Anthropic's extended thinking allows Claude to reason before responding. With interleaved thinking, Claude can think between tool calls for more sophisticated multi-step reasoning.

from thinllm import llm, LLMConfig, ModelParams, ThinkingConfig, UserMessage

def get_weather(location: str) -> str:
    """Get weather for a location."""
    return f"Weather in {location}: Sunny, 72°F"

def get_time(timezone: str) -> str:
    """Get time for a timezone."""
    return f"Time in {timezone}: 2:30 PM"

# Configure with interleaved thinking
config = LLMConfig(
    provider="anthropic",
    model_id="claude-sonnet-4-5",
    params=ModelParams(
        max_output_tokens=4096,
        temperature=1.0,  # Required for thinking
        thinking=ThinkingConfig(
            enabled=True,
            thinking_budget=10000,  # Can exceed max_output_tokens with interleaved
            anthropic_interleaved_thinking=True,  # Enable interleaved thinking
        ),
    ),
)

messages = [
    UserMessage(content="What's the weather in Paris, and what time is it there?")
]

# Claude will think between tool calls
response = llm(config, messages, tools=[get_weather, get_time])

Key Features:

• Interleaved Thinking: Claude can think between tool calls (set anthropic_interleaved_thinking=True)
• Flexible Token Budget: With interleaved thinking, thinking_budget can exceed max_output_tokens
• Temperature Requirement: Extended thinking requires temperature=1.0
• Beta API: Automatically uses the beta API when interleaved thinking is enabled
• Bedrock Compatible: Works with both Anthropic and Bedrock Anthropic providers

When to Use:

• Complex multi-step problems requiring planning
• Tool use scenarios with multiple steps
• Tasks that benefit from reflection between actions

See examples/anthropic_interleaved_thinking_example.py for comprehensive examples.

AWS Bedrock (Anthropic Models)

Use Claude models through AWS Bedrock:

config = LLMConfig(
    provider="bedrock_anthropic",
    model_id="us.anthropic.claude-sonnet-4-5-20250929-v1:0",
    params=ModelParams(
        temperature=0.0,
        max_output_tokens=1024
    )
)

Supported Models: Any Anthropic model available in AWS Bedrock

• us.anthropic.claude-sonnet-4-5-20250929-v1:0
• global.anthropic.claude-sonnet-4-5-20250929-v1:0
• And other Bedrock model IDs

Setup: Configure AWS credentials through environment variables or AWS CLI:

AWS_ACCESS_KEY_ID=your-access-key
AWS_SECRET_ACCESS_KEY=your-secret-key
AWS_REGION=us-east-1  # or your preferred region

Google Gemini

config = LLMConfig(
    provider="gemini",
    model_id="gemini-2.5-flash",
    params=ModelParams(temperature=0.7)
)

Supported Models: gemini-2.5-flash, gemini-2.5-pro, gemini-1.5-pro, etc.

Setup: Set GEMINI_API_KEY environment variable

Provider Feature Comparison

Feature	OpenAI	Azure OpenAI	Anthropic	Bedrock	Gemini
Basic Chat	✅	✅	✅	✅	✅
Streaming	✅	✅	✅	✅	✅
Structured Output	✅	✅	✅	✅	✅
Function Calling	✅	✅	✅	✅	✅
Vision (Images)	✅	✅	✅	✅	✅
Prompt Caching	❌	❌	✅	✅	❌
Thinking Mode	❌	❌	❌	❌	✅
Built-in Search	❌	❌	❌	❌	✅
Code Execution	❌	❌	❌	❌	✅
Entra ID Auth	❌	✅	❌	❌	❌

Gemini-Specific Features

Thinking Mode (Extended Reasoning):

config = LLMConfig(
    provider="gemini",
    model_id="gemini-2.5-pro",
    params=ModelParams(temperature=0.7),
    model_args={
        "thinking_budget": 2048,      # Allocate tokens for reasoning
        "include_thoughts": True,     # Include reasoning in response
    }
)

Built-in Tools:

# Google Search
response = llm(config, messages, tools=[{"google_search": {}}])

# Code Execution
response = llm(config, messages, tools=[{"code_execution": {}}])

# URL Context
response = llm(config, messages, tools=[{"url_context": {}}])

API Reference

Main Functions

llm()

Unified interface for LLM interactions.

def llm(
    llm_config: LLMConfig,
    messages: list[MessageType],
    *,
    output_schema: type[OutputSchemaType] | None = None,
    tools: list[Tool | Callable | dict] | None = None,
    stream: bool = False,
) -> AIMessage | OutputSchemaType | Generator[...]:
    """
    Make an LLM request.
    
    Args:
        llm_config: Configuration for the LLM
        messages: List of conversation messages
        output_schema: Optional Pydantic model for structured output
        tools: Optional list of tools/functions
        stream: Whether to stream the response
        
    Returns:
        AIMessage, structured output, or generator
    """

Configuration

LLMConfig

class LLMConfig(BaseModel):
    provider: str                           # "openai", "azure_openai", "anthropic", "bedrock_anthropic", or "gemini"
    model_id: str                          # Model identifier (e.g., "gpt-4", "claude-sonnet-4")
    params: ModelParams | None = None      # Standard model parameters
    credentials: Credentials | None = None # Optional credentials (required for Azure OpenAI)
    model_args: dict[str, Any] = {}        # Provider-specific arguments

ModelParams

Standard parameters that work across all providers:

class ModelParams(BaseModel):
    temperature: float | None = None           # Controls randomness (0.0 to 2.0)
    max_output_tokens: int | None = None       # Maximum tokens to generate
    top_p: float | None = None                 # Nucleus sampling parameter
    top_k: int | None = None                   # Top-k sampling parameter
    stop_sequences: list[str] | None = None    # Sequences where the model stops

Messages

• SystemMessage(content: str) - System instructions
• UserMessage(content: str | list[ContentBlock]) - User input
• AIMessage(content: str | list[ContentBlock]) - AI response

Content Blocks

• InputTextBlock - Text input from user
• OutputTextBlock - Text output from AI
• InputImageBlock - Image input
• ReasoningContent - Reasoning/thinking content
• ToolCallContent - Tool call request
• ToolResultContent - Tool execution result

Examples

See the examples/ directory for complete examples:

• azure_openai_example.py: Azure OpenAI with API Key and Microsoft Entra ID authentication
• gemini_example.py: Comprehensive Gemini provider examples with thinking mode
• bedrock_example.py: AWS Bedrock integration examples
• agent_example.py: Multi-turn conversations and tool usage patterns
• streamlit_streaming_chat.py: Interactive streaming chat interface
• streamlit_agent_chat.py: Web-based chat interface with debug view

Running Examples

# Install example dependencies
pip install streamlit

# Run a specific example
python examples/bedrock_example.py

# Run the Streamlit chat interface
streamlit run examples/streamlit_streaming_chat.py

Troubleshooting

Common Issues

Import Errors

# Ensure you've installed the provider-specific dependencies
pip install thinllm[anthropic]  # or openai, gemini, bedrock

API Key Issues

# Make sure your environment variables are set
from dotenv import load_dotenv
load_dotenv()

# Or set them directly
import os
os.environ["ANTHROPIC_API_KEY"] = "your-key"

AWS Bedrock Authentication

# Configure AWS CLI (recommended)
aws configure

# Or set environment variables
export AWS_ACCESS_KEY_ID=your-key
export AWS_SECRET_ACCESS_KEY=your-secret
export AWS_REGION=us-east-1

Model Not Found

# Bedrock models use region-specific IDs:
# "us.anthropic.claude-sonnet-4-5-20250929-v1:0"  # US region
# "global.anthropic.claude-sonnet-4-5-20250929-v1:0"  # Global

Roadmap

The following features are planned for ThinLLM. These are up for grabs - contributions are welcome!

Core API Features

• Async API Support: Asynchronous API for non-blocking LLM interactions

Provider Enhancements

• Vertex AI Support: Integration with Google Cloud's Vertex AI platform
• Anthropic Caching: Prompt caching support for improved performance and cost efficiency
• Anthropic Extended Thinking Support: Enhanced reasoning capabilities with interleaved thinking
• Anthropic Support for Thinking with Structured Output: Using beta API where applicable

Multimodal Capabilities

• Image Generation Support: Native support for image generation across providers

Tool & Function Capabilities

• Auto Function Call Support: Automatic execution of function calls
• MCP Support: Model Context Protocol integration
• Computer Use Support: Anthropic's computer use capabilities

Built-in Server Tools

• Search Tool: Web search capabilities
• Fetch Tool: URL fetching and content retrieval
• Code Executor: Safe code execution environment

Provider-Specific Tools

Anthropic Tools

• Tool Search Tool: Anthropic's search tool integration
• Memory Tool: Persistent memory capabilities
• Text Editor Tool: Advanced text editing
• Bash Tool: Command-line execution

Gemini Tools

• Google Maps Integration: Location and mapping features
• URL Context: Enhanced URL processing

Response Features

• Citation Support: Source attribution and citation tracking
• Raw Response in AI Message: Access to raw API responses
• Add stop reason to the response: Add stop reason to the AIMessage

Additional Capabilities

• Embedding Support: Text embedding generation across providers
• Observability Platforms:
  • Langfuse integration
  • Langsmith integration
  • Additional observability platforms

Contributing

Contributions are welcome! Please see CONTRIBUTING.md for detailed guidelines on:

• Setting up the development environment
• Running tests
• Code quality standards
• Common development patterns
• Submitting pull requests

License

MIT License - see LICENSE file for details

Acknowledgments

• Built with Pydantic for data validation
• Uses official SDKs: openai, anthropic, google-genai