Octo AI Python Library

The Octo AI Python Library provides convenient access to the Octo AI API from applications written in Python.

Documentation

API reference documentation is available here.

Whisper is not currently available. If you're interested in using speech-to-text models, please contact us at customer-success@octo.ai

Installation

Add this dependency to your project's build file:

pip install octoai
# or
poetry add octoai

Usage

Simply import OctoAI and start making calls to our API.

from octoai.client import OctoAI
from octoai.text_gen import ChatMessage

client = OctoAI(
    api_key="YOUR_API_KEY"  # Defaults to your OCTOAI_TOKEN environment variable
)
client.text_gen.create_chat_completion(
    messages=[
        ChatMessage(
            role="role",
        )
    ],
    model="model",
)

Async Client

The SDK also exports an async client so that you can make non-blocking calls to our API.

from octoai.client import AsyncOctoAI
from octoai.text_gen import ChatMessage

client = AsyncOctoAI(
  api_key="YOUR_API_KEY" # Defaults to your OCTOAI_TOKEN environment variable
)

async def main() -> None:
    await client.text_gen.create_chat_completion(
        messages=[
            ChatMessage(
                role="role",
            )
        ],
        model="model",
    )

asyncio.run(main())

OctoAI Services

The client nests key services within the provisioned client:

from octoai.client import OctoAI

client = OctoAI(...)

# The image generation service
client.image_gen.generate_sd(...)

# The text generation service
client.text_gen.create_chat_completion(...)

# The asset library service
client.asset_library.create(...)

# The fine-tuning service
client.fine_tuning.create(...)

Exception Handling

All errors thrown by the SDK will be subclasses of ApiError.

import octoai

try:
  client.text_gen.create_chat_completion(...)
except octoai.core.ApiError as e: # Handle all errors
  print(e.status_code)
  print(e.body)

Advanced

Retries

The OctoAI SDK is instrumented with automatic retries with exponential backoff. A request will be retried as long as the request is deemed retriable and the number of retry attempts has not grown larger than the configured retry limit.

A request is deemed retriable when any of the following HTTP status codes is returned:

• 408 (Timeout)
• 409 (Conflict)
• 429 (Too Many Requests)
• 5XX (Internal Server Errors)

Use the max_retries request option to configure this behavior.

from octoai.client import OctoAI

client = OctoAI(...)

# Override retries for a specific method
client.text_gen.create_chat_completion(..., {
    max_retries=5
})

Timeouts

By default, requests time out after 60 seconds. You can configure this with a timeout option at the client or request level.

from octoai.client import OctoAI

client = OctoAI(
    # All timeouts are 20 seconds
    timeout=20.0,
)

# Override timeout for a specific method
client.text_gen.create_chat_completion(..., {
    timeout_in_seconds=20.0
})

Custom HTTP client

You can override the httpx client to customize it for your use-case. Some common use-cases include support for proxies and transports.

import httpx

from octoai.client import OctoAI

client = OctoAI(
    http_client=httpx.Client(
        proxies="http://my.test.proxy.example.com",
        transport=httpx.HTTPTransport(local_address="0.0.0.0"),
    ),
)

Beta Status

This SDK is in beta, and there may be breaking changes between versions without a major version update. Therefore, we recommend pinning the package version to a specific version. This way, you can install the same version each time without breaking changes.

Contributing

While we value open-source contributions to this SDK, this library is generated programmatically. Additions made directly to this library would have to be moved over to our generation code, otherwise they would be overwritten upon the next generated release. Feel free to open a PR as a proof of concept, but know that we will not be able to merge it as-is. We suggest opening an issue first to discuss with us!

On the other hand, contributions to the README are always very welcome!