ai21 2.2.0rc1

No project description provided

AI21 Labs Python SDK

---

Migration from v1.3.4 and below

In v2.0.0 we introduced a new SDK that is not backwards compatible with the previous version. This version allows for Non static instances of the client, defined parameters to each resource, modelized responses and more.

Migration Examples

Instance creation (not available in v1.3.4 and below)

from ai21 import AI21Client

client = AI21Client(api_key='my_api_key')

# or set api_key in environment variable - AI21_API_KEY and then
client = AI21Client()

We No longer support static methods for each resource, instead we have a client instance that has a method for each allowing for more flexibility and better control.

Completion before/after

prompt = "some prompt"

- import ai21
- response = ai21.Completion.execute(model="j2-light", prompt=prompt, maxTokens=2)

+ from ai21 import AI21Client
+ client = ai21.AI21Client()
+ response = client.completion.create(model="j2-light", prompt=prompt, max_tokens=2)

This applies to all resources. You would now need to create a client instance and use it to call the resource method.

Tokenization and Token counting before/after

- response = ai21.Tokenization.execute(text=prompt)
- print(len(response)) # number of tokens

+ from ai21 import AI21Client
+ client = AI21Client()
+ token_count = client.count_tokens(text=prompt)

Key Access in Response Objects before/after

It is no longer possible to access the response object as a dictionary. Instead, you can access the response object as an object with attributes.

- import ai21
- response = ai21.Summarize.execute(source="some text", sourceType="TEXT")
- response["summary"]

+ from ai21 import AI21Client
+ from ai21.models import DocumentType
+ client = AI21Client()
+ response = client.summarize.create(source="some text", source_type=DocumentType.TEXT)
+ response.summary

---

AWS Client Creations

Bedrock Client creation before/after

- import ai21
- destination = ai21.BedrockDestination(model_id=ai21.BedrockModelID.J2_MID_V1)
- response = ai21.Completion.execute(prompt=prompt, maxTokens=1000, destination=destination)

+ from ai21 import AI21BedrockClient, BedrockModelID
+ client = AI21BedrockClient()
+ response = client.completion.create(prompt=prompt, max_tokens=1000, model_id=BedrockModelID.J2_MID_V1)

SageMaker Client creation before/after

- import ai21
- destination = ai21.SageMakerDestination("j2-mid-test-endpoint")
- response = ai21.Completion.execute(prompt=prompt, maxTokens=1000, destination=destination)

+ from ai21 import AI21SageMakerClient
+ client = AI21SageMakerClient(endpoint_name="j2-mid-test-endpoint")
+ response = client.completion.create(prompt=prompt, max_tokens=1000)

Installation

pip

pip install ai21

Usage

Model Types

Wherever you are required to pass a model name as a parameter, you can use any of the available AI21 state-of-the-art models:

• j2-light
• j2-mid
• j2-ultra

you can read more about the models here.

---

Client Instance Creation

from ai21 import AI21Client

client = AI21Client(
    # defaults to os.enviorn.get('AI21_API_KEY')
    api_key='my_api_key',
)

response = client.completion.create(
    prompt="<your prompt here>",
    max_tokens=10,
    model="j2-mid",
    temperature=0.3,
    top_p=1,
)

print(response.completions)
print(response.prompt)

Token Counting

---

By using the count_tokens method, you can estimate the billing for a given request.

from ai21 import AI21Client

client = AI21Client()
client.count_tokens(text="some text")  # returns int

File Upload

---

from ai21 import AI21Client

client = AI21Client()

file_id = client.library.files.create(
    file_path="path/to/file",
    path="path/to/file/in/library",
    labels=["label1", "label2"],
    public_url="www.example.com",
)

uploaded_file = client.library.files.get(file_id)

Environment Variables

---

You can set several environment variables to configure the client.

Logging

We use the standard library logging module.

To enable logging, set the AI21_LOG_LEVEL environment variable.

$ export AI21_LOG_LEVEL=debug

Other Important Environment Variables

• AI21_API_KEY - Your API key. If not set, you must pass it to the client constructor.
• AI21_API_VERSION - The API version. Defaults to v1.
• AI21_API_HOST - The API host. Defaults to https://api.ai21.com/v1/.
• AI21_TIMEOUT_SEC - The timeout for API requests.
• AI21_NUM_RETRIES - The maximum number of retries for API requests. Defaults to 3 retries.
• AI21_AWS_REGION - The AWS region to use for AWS clients. Defaults to us-east-1.

Error Handling

---

from ai21 import errors as ai21_errors
from ai21 import AI21Client, AI21APIError
from ai21.models import ChatMessage

client = AI21Client()

system = "You're a support engineer in a SaaS company"
messages = [
        # Notice the given role does not exist and will be the reason for the raised error
        ChatMessage(text="Hello, I need help with a signup process.", role="Non-Existent-Role"),
    ]

try:
    chat_completion = client.chat.create(
        messages=messages,
        model="j2-ultra",
        system=system
    )
except ai21_errors.AI21ServerError as e:
    print("Server error and could not be reached")
    print(e.details)
except ai21_errors.TooManyRequestsError as e:
    print("A 429 status code was returned. Slow down on the requests")
except AI21APIError as e:
    print("A non 200 status code error. For more error types see ai21.errors")

AWS Clients

---

AI21 Library provides convenient ways to interact with two AWS clients for use with AWS SageMaker and AWS Bedrock.

Installation

---

pip install "ai21[AWS]"

This will make sure you have the required dependencies installed, including boto3 >= 1.28.82.

Usage

---

SageMaker

from ai21 import AI21SageMakerClient

client = AI21SageMakerClient(endpoint_name="j2-endpoint-name")
response = client.summarize.create(
    source="Text to summarize",
    source_type="TEXT",
)
print(response.summary)

With Boto3 Session

from ai21 import AI21SageMakerClient
import boto3
boto_session = boto3.Session(region_name="us-east-1")

client = AI21SageMakerClient(
    session=boto_session,
    endpoint_name="j2-endpoint-name",
)

Bedrock

---

from ai21 import AI21BedrockClient, BedrockModelID

client = AI21BedrockClient(region='us-east-1') # region is optional, as you can use the env variable instead
response = client.completion.create(
    prompt="Your prompt here",
    model_id=BedrockModelID.J2_MID_V1,
    max_tokens=10,
)
print(response.completions[0].data.text)

With Boto3 Session

from ai21 import AI21BedrockClient, BedrockModelID
import boto3
boto_session = boto3.Session(region_name="us-east-1")

client = AI21BedrockClient(
    session=boto_session,
)

response = client.completion.create(
    prompt="Your prompt here",
    model_id=BedrockModelID.J2_MID_V1,
    max_tokens=10,
)