aisuite-async

Simple, unified interface to multiple Generative AI providers.

aisuite-async makes it easy for developers to use multiple LLM through a standardized interface. Using an interface similar to OpenAI's, aisuite-async makes it easy to interact with the most popular LLMs and compare the results. It is a thin wrapper around python client libraries, and allows creators to seamlessly swap out and test responses from different LLM providers without changing their code. Today, the library is primarily focussed on chat completions. We will expand it cover more use cases in near future.

aisuite-async is a temporary fork of aisuite with async implementation, created until the original aisuite library gets async capabilities. Support, please, the pull-request https://github.com/andrewyng/aisuite/pull/185.

======= Currently supported providers are:

• Anthropic
• AWS
• Azure
• Cerebras
• Google
• Groq
• HuggingFace Ollama
• Mistral
• OpenAI
• Sambanova
• Watsonx

Async support is currently available for:

• OpenAI
• Anthropic
• Mistral
• Fireworks.

To maximize stability, aisuite-async uses either the HTTP endpoint or the SDK for making calls to the provider.

Installation

You can install just the base aisuite-async package, or install a provider's package along with aisuite-async.

This installs just the base package without installing any provider's SDK.

pip install aisuite-async

This installs aisuite-async along with anthropic's library.

pip install 'aisuite-async[anthropic]'

This installs all the provider-specific libraries

pip install 'aisuite-async[all]'

Set up

To get started, you will need API Keys for the providers you intend to use. You'll need to install the provider-specific library either separately or when installing aisuite-async.

The API Keys can be set as environment variables, or can be passed as config to the Client or AsyncClient constructor. You can use tools like python-dotenv or direnv to set the environment variables manually. Please take a look at the examples folder to see usage.

Here is a short example of using aisuite-async to generate chat completion responses from gpt-4o and claude-3-5-sonnet.

Set the API keys.

export OPENAI_API_KEY="your-openai-api-key"
export ANTHROPIC_API_KEY="your-anthropic-api-key"

Use the python client.

import aisuite_async as ai
client = ai.Client()

models = ["openai:gpt-4o", "anthropic:claude-3-5-sonnet-20240620"]

messages = [
    {"role": "system", "content": "Respond in Pirate English."},
    {"role": "user", "content": "Tell me a joke."},
]

for model in models:
    response = client.chat.completions.create(
        model=model,
        messages=messages,
        temperature=0.75
    )
    print(response.choices[0].message.content)

Use the async client.

import aisuite_async as ai
import asyncio

client = ai.AsyncClient()

models = ["openai:gpt-4o", "anthropic:claude-3-5-sonnet-20240620"]

messages = [
    {"role": "system", "content": "Respond in Pirate English."},
    {"role": "user", "content": "Tell me a joke."},
]

responses = await asyncio.gather(*[client.chat.completions.create(
        model=model,
        messages=messages,
        temperature=0.75
    )
    for model in models
])

for response in responses:
    print(response.choices[0].message.content)

Note that the model name in the create() call uses the format - <provider>:<model-name>. aisuite-async will call the appropriate provider with the right parameters based on the provider value. For a list of provider values, you can look at the directory - aisuite_async/providers/. The list of supported providers are of the format - <provider>_provider.py in that directory. We welcome providers adding support to this library by adding an implementation file in this directory. Please see section below for how to contribute.

For more examples, check out the examples directory where you will find several notebooks that you can run to experiment with the interface.

Adding support for a provider

We have made easy for a provider or volunteer to add support for a new platform.

Naming Convention for Provider Modules

We follow a convention-based approach for loading providers, which relies on strict naming conventions for both the module name and the class name. The format is based on the model identifier in the form provider:model.

• The provider's module file must be named in the format <provider>_provider.py.
• The class inside this module must follow the format: the provider name with the first letter capitalized, followed by the suffix Provider.

Examples

• Hugging Face: The provider class should be defined as:

  class HuggingfaceProvider(BaseProvider)
  

  in providers/huggingface_provider.py.

• OpenAI: The provider class should be defined as:

  class OpenaiProvider(BaseProvider)
  

  in providers/openai_provider.py

This convention simplifies the addition of new providers and ensures consistency across provider implementations.

Tool Calling

aisuite-async provides a simple abstraction for tool/function calling that works across supported providers. This is in addition to the regular abstraction of passing JSON spec of the tool to the tools parameter. The tool calling abstraction makes it easy to use tools with different LLMs without changing your code.

There are two ways to use tools with aisuite-async:

1. Manual Tool Handling

This is the default behavior when max_turns is not specified. You can pass tools in the OpenAI tool format:

def will_it_rain(location: str, time_of_day: str):
    """Check if it will rain in a location at a given time today.
    
    Args:
        location (str): Name of the city
        time_of_day (str): Time of the day in HH:MM format.
    """
    return "YES"

tools = [{
    "type": "function",
    "function": {
        "name": "will_it_rain",
        "description": "Check if it will rain in a location at a given time today",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "Name of the city"
                },
                "time_of_day": {
                    "type": "string",
                    "description": "Time of the day in HH:MM format."
                }
            },
            "required": ["location", "time_of_day"]
        }
    }
}]

response = client.chat.completions.create(
    model="openai:gpt-4o",
    messages=messages,
    tools=tools
)

2. Automatic Tool Execution

When max_turns is specified, you can pass a list of callable Python functions as the tools parameter. aisuite-async will automatically handle the tool calling flow:

def will_it_rain(location: str, time_of_day: str):
    """Check if it will rain in a location at a given time today.
    
    Args:
        location (str): Name of the city
        time_of_day (str): Time of the day in HH:MM format.
    """
    return "YES"

client = ai.Client()
messages = [{
    "role": "user",
    "content": "I live in San Francisco. Can you check for weather "
               "and plan an outdoor picnic for me at 2pm?"
}]

# Automatic tool execution with max_turns
response = client.chat.completions.create(
    model="openai:gpt-4o",
    messages=messages,
    tools=[will_it_rain],
    max_turns=2  # Maximum number of back-and-forth tool calls
)
print(response.choices[0].message.content)

When max_turns is specified, aisuite-async will:

1. Send your message to the LLM
2. Execute any tool calls the LLM requests
3. Send the tool results back to the LLM
4. Repeat until the conversation is complete or max_turns is reached

In addition to response.choices[0].message, there is an additional field response.choices[0].intermediate_messages: which contains the list of all messages including tool interactions used. This can be used to continue the conversation with the model. For more detailed examples of tool calling, check out the examples/tool_calling_abstraction.ipynb notebook.

License

aisuite-async is released under the MIT License. You are free to use, modify, and distribute the code for both commercial and non-commercial purposes.

Contributing

If you would like to contribute, please read our Contributing Guide!