structured outputs for llm
Project description
Instructor: Structured LLM Outputs
Instructor is a Python library that makes it a breeze to work with structured outputs from large language models (LLMs). Built on top of Pydantic, it provides a simple, transparent, and user-friendly API to manage validation, retries, and streaming responses. Get ready to supercharge your LLM workflows!
Want your logo on our website?
If your company use instructor a lot, we'd love to have your logo on our website! Please fill out this form
Key Features
- Response Models: Specify Pydantic models to define the structure of your LLM outputs
- Retry Management: Easily configure the number of retry attempts for your requests
- Validation: Ensure LLM responses conform to your expectations with Pydantic validation
- Streaming Support: Work with Lists and Partial responses effortlessly
- Flexible Backends: Seamlessly integrate with various LLM providers beyond OpenAI
- Support in many Languages: We support many languages including Python, TypeScript, Ruby, Go, and Elixir
Get Started in Minutes
Install Instructor with a single command:
pip install -U instructor
Now, let's see Instructor in action with a simple example:
import instructor
from pydantic import BaseModel
from openai import OpenAI
# Define your desired output structure
class UserInfo(BaseModel):
name: str
age: int
# Patch the OpenAI client
client = instructor.from_openai(OpenAI())
# Extract structured data from natural language
user_info = client.chat.completions.create(
model="gpt-3.5-turbo",
response_model=UserInfo,
messages=[{"role": "user", "content": "John Doe is 30 years old."}],
)
print(user_info.name)
#> John Doe
print(user_info.age)
#> 30
Using Anthropic Models
import instructor
from anthropic import Anthropic
from pydantic import BaseModel
class User(BaseModel):
name: str
age: int
client = instructor.from_anthropic(Anthropic())
# note that client.chat.completions.create will also work
resp = client.messages.create(
model="claude-3-opus-20240229",
max_tokens=1024,
system="You are a world class AI that excels at extracting user data from a sentence",
messages=[
{
"role": "user",
"content": "Extract Jason is 25 years old.",
}
],
response_model=User,
)
assert isinstance(resp, User)
assert resp.name == "Jason"
assert resp.age == 25
Using Cohere Models
Make sure to install cohere
and set your system environment variable with export CO_API_KEY=<YOUR_COHERE_API_KEY>
.
pip install cohere
import instructor
import cohere
from pydantic import BaseModel
class User(BaseModel):
name: str
age: int
client = instructor.from_cohere(cohere.Client())
# note that client.chat.completions.create will also work
resp = client.chat.completions.create(
model="command-r-plus",
max_tokens=1024,
messages=[
{
"role": "user",
"content": "Extract Jason is 25 years old.",
}
],
response_model=User,
)
assert isinstance(resp, User)
assert resp.name == "Jason"
assert resp.age == 25
Using Gemini Models
Make sure you install the Google AI Python SDK. You should set a GOOGLE_API_KEY
environment variable with your API key.
Gemini tool calling also requires jsonref
to be installed.
pip install google-generativeai jsonref
import instructor
import google.generativeai as genai
from pydantic import BaseModel
class User(BaseModel):
name: str
age: int
# genai.configure(api_key=os.environ["API_KEY"]) # alternative API key configuration
client = instructor.from_gemini(
client=genai.GenerativeModel(
model_name="models/gemini-1.5-flash-latest", # model defaults to "gemini-pro"
),
mode=instructor.Mode.GEMINI_JSON,
)
Alternatively, you can call Gemini from the OpenAI client.You'll have to setup gcloud
, get setup on Vertex AI, and install the Google Auth library.
pip install google-auth
import google.auth
import google.auth.transport.requests
import instructor
from openai import OpenAI
from pydantic import BaseModel
creds, project = google.auth.default()
auth_req = google.auth.transport.requests.Request()
creds.refresh(auth_req)
# Pass the Vertex endpoint and authentication to the OpenAI SDK
PROJECT = 'PROJECT_ID'
LOCATION = (
'LOCATION' # https://cloud.google.com/vertex-ai/generative-ai/docs/learn/locations
)
base_url = f'https://{LOCATION}-aiplatform.googleapis.com/v1beta1/projects/{PROJECT}/locations/{LOCATION}/endpoints/openapi'
client = instructor.from_openai(
OpenAI(base_url=base_url, api_key=creds.token), mode=instructor.Mode.JSON
)
# JSON mode is req'd
class User(BaseModel):
name: str
age: int
resp = client.chat.completions.create(
model="google/gemini-1.5-flash-001",
max_tokens=1024,
messages=[
{
"role": "user",
"content": "Extract Jason is 25 years old.",
}
],
response_model=User,
)
assert isinstance(resp, User)
assert resp.name == "Jason"
assert resp.age == 25
Using Litellm
import instructor
from litellm import completion
from pydantic import BaseModel
class User(BaseModel):
name: str
age: int
client = instructor.from_litellm(completion)
resp = client.chat.completions.create(
model="claude-3-opus-20240229",
max_tokens=1024,
messages=[
{
"role": "user",
"content": "Extract Jason is 25 years old.",
}
],
response_model=User,
)
assert isinstance(resp, User)
assert resp.name == "Jason"
assert resp.age == 25
Types are inferred correctly
This was the dream of instructor but due to the patching of openai, it wasnt possible for me to get typing to work well. Now, with the new client, we can get typing to work well! We've also added a few create_*
methods to make it easier to create iterables and partials, and to access the original completion.
Calling create
import openai
import instructor
from pydantic import BaseModel
class User(BaseModel):
name: str
age: int
client = instructor.from_openai(openai.OpenAI())
user = client.chat.completions.create(
model="gpt-4-turbo-preview",
messages=[
{"role": "user", "content": "Create a user"},
],
response_model=User,
)
Now if you use an IDE, you can see the type is correctly inferred.
Handling async: await create
This will also work correctly with asynchronous clients.
import openai
import instructor
from pydantic import BaseModel
client = instructor.from_openai(openai.AsyncOpenAI())
class User(BaseModel):
name: str
age: int
async def extract():
return await client.chat.completions.create(
model="gpt-4-turbo-preview",
messages=[
{"role": "user", "content": "Create a user"},
],
response_model=User,
)
Notice that simply because we return the create
method, the extract()
function will return the correct user type.
Returning the original completion: create_with_completion
You can also return the original completion object
import openai
import instructor
from pydantic import BaseModel
client = instructor.from_openai(openai.OpenAI())
class User(BaseModel):
name: str
age: int
user, completion = client.chat.completions.create_with_completion(
model="gpt-4-turbo-preview",
messages=[
{"role": "user", "content": "Create a user"},
],
response_model=User,
)
Streaming Partial Objects: create_partial
In order to handle streams, we still support Iterable[T]
and Partial[T]
but to simplify the type inference, we've added create_iterable
and create_partial
methods as well!
import openai
import instructor
from pydantic import BaseModel
client = instructor.from_openai(openai.OpenAI())
class User(BaseModel):
name: str
age: int
user_stream = client.chat.completions.create_partial(
model="gpt-4-turbo-preview",
messages=[
{"role": "user", "content": "Create a user"},
],
response_model=User,
)
for user in user_stream:
print(user)
#> name=None age=None
#> name=None age=None
#> name=None age=None
#> name=None age=None
#> name=None age=None
#> name=None age=None
#> name='John Doe' age=None
#> name='John Doe' age=None
#> name='John Doe' age=None
#> name='John Doe' age=30
#> name='John Doe' age=30
# name=None age=None
# name='' age=None
# name='John' age=None
# name='John Doe' age=None
# name='John Doe' age=30
Notice now that the type inferred is Generator[User, None]
Streaming Iterables: create_iterable
We get an iterable of objects when we want to extract multiple objects.
import openai
import instructor
from pydantic import BaseModel
client = instructor.from_openai(openai.OpenAI())
class User(BaseModel):
name: str
age: int
users = client.chat.completions.create_iterable(
model="gpt-4-turbo-preview",
messages=[
{"role": "user", "content": "Create 2 users"},
],
response_model=User,
)
for user in users:
print(user)
#> name='John Doe' age=30
#> name='Jane Doe' age=28
# User(name='John Doe', age=30)
# User(name='Jane Smith', age=25)
Evals
We invite you to contribute to evals in pytest
as a way to monitor the quality of the OpenAI models and the instructor
library. To get started check out the evals for anthropic and OpenAI and contribute your own evals in the form of pytest tests. These evals will be run once a week and the results will be posted.
Contributing
If you want to help, checkout some of the issues marked as good-first-issue
or help-wanted
found here. They could be anything from code improvements, a guest blog post, or a new cookbook.
CLI
We also provide some added CLI functionality for easy convinience:
-
instructor jobs
: This helps with the creation of fine-tuning jobs with OpenAI. Simple useinstructor jobs create-from-file --help
to get started creating your first fine-tuned GPT3.5 model -
instructor files
: Manage your uploaded files with ease. You'll be able to create, delete and upload files all from the command line -
instructor usage
: Instead of heading to the OpenAI site each time, you can monitor your usage from the cli and filter by date and time period. Note that usage often takes ~5-10 minutes to update from OpenAI's side
License
This project is licensed under the terms of the MIT License.
Contributors
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for instructor-1.5.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | da25abbf1ab792fb094992f1d9ce593e26fe458cb1f9a8e7ebf9d68f3f2c757a |
|
MD5 | 8a95814557683e0620b7b07a2da4b1a1 |
|
BLAKE2b-256 | 0e945f96c1d87ba732ba6b2a9751314ddb69bfcd8d05ed0f7e93ee17fd4df068 |