Wrapper library for openai to send events to Libretto
Project description
Libretto OpenAI Client
A drop-in replacement of openai.Client
for sending events to Libretto.
Features
- Provides a patched
openai.Client
that allows for setting Libretto-specific parameters for each request. - Currently supports the synchronous versions of
completions.create()
andchat.completions.create()
.
Get Started
To send events to Libretto, you'll need to create a project. From the project you'll need two things:
- API key: (
api_key
) This is generated for the project and is used to identify the project and environment (dev, staging, prod) that the event is coming from. - Template Name: (
prompt_template_name
) This uniquely identifies a particular prompt that you are using and allows projects to have multiple prompts. This can be in any format but we recommend using a dash-separated format, e.g.my-prompt-name
.
Note: Prompt template names can be auto-generated if the allow_unnamed_prompts
configuration option is set (see below). However, if you rely on auto-generated names, new revisions of the same prompt will show up as different prompt templates in Libretto.
Usage
You can use the libretto_openai.Client
anywhere that you're currently using the official openai.Client
.
When instantiating a libretto_openai.Client
, you can/should provide any of the existing openai.Client
parameters in the constructor. Libretto-specific configuration can be provided via an additional libretto
argument (see below).
To allow our tools to separate the "prompt" from the "prompt parameters", use TemplateChat
and TemplateText
to create templates.
Use TemplateChat
For the ChatCompletion APIs:
from libretto_openai import (
Client,
LibrettoConfig,
LibrettoCreateParams,
TemplateChat,
)
client = Client(
api_key="<OpenAI API Key>",
libretto=LibrettoConfig(
api_key="<Libretto API Key>",
),
)
completion = client.chat.completions.create(
# Standard OpenAI parameters
model="gpt-3.5-turbo",
messages=TemplateChat(
[{"role": "user", "content": "Show me an emoji that matches the sport: {sport}"}],
{"sport": "soccer"},
),
libretto=LibrettoCreateParams(
prompt_template_name="sport-emoji",
),
)
Use TemplateText
for the Completion API:
from libretto_openai import (
Client,
LibrettoConfig,
LibrettoCreateParams,
TemplateChat,
)
client = Client(
api_key="<OpenAI API Key>",
libretto=LibrettoConfig(
api_key="<Libretto API Key>",
),
)
completion = client.completions.create(
# Standard OpenAI parameters
model="text-davinci-003",
prompt=TemplateText(
"Show me an emoji that matches the sport: {sport}",
{"sport": "soccer"},
),
libretto=LibrettoCreateParams(
prompt_template_name="sport-emoji",
),
)
Advanced usage
Manually passing parameters
While the use of TemplateText
and TemplateChat
are preferred, you can optionally specify template data inline when calling the create()
method:
completion = openai.ChatCompletion.create(
model="gpt-3.5-turbo",
# Note we are passing the raw messages object here
messages=[{"role": "user", "content": "Show me an emoji that matches the sport: soccer"}],
libretto=LibrettoCreateParams(
# call configuration
prompt_template_name="sport-emoji",
# Here the prompt and parameters are passed seperately
template_params={"sport": "soccer"},
template_chat=[
{"role": "user", "content": "Show me an emoji that matches the sport: {sport}"}
],
),
)
Configuration
The libretto
kwarg that's present on the Client
constructor is a LibrettoConfig
object with the following options:
prompt_template_name
: A default name to associate with prompts. If provided, this is the name that will be associated with anycreate
call that's made without alibretto.prompt_template_name
parameter.allow_unnamed_prompts
: When set toTrue
, every prompt will be sent to Libretto even if no prompt template name as been provided (either via theprompt_template_name
kwarg or via thelibretto.prompt_template_name
parameter oncreate
).False
by default.redact_pii
: WhenTrue
, certain personally identifying information (PII) will be attempted to be redacted before being sent to the Libretto backend. See thepii
package for details about the types of PII being detected/redacted.False
by default.
Additional Create Call Parameters
When calling create()
, a libretto
argument should be provided to give Libretto-specific context to the call. The following parameters maybe specified:
-
chat_id
: The id of a "chat session" - if the chat API is being used in a conversational context, then the same chat id can be provided so that the events are grouped together, in order. If not provided, this will be left blank. -
template_chat
: The chat template to record for chat requests. This is a list of dictionaries with the following keys:role
: The role of the speaker. Either"system"
,"user"
or"ai"
.content
: The content of the message. This can be a string or a template string with{}
placeholders.
For example:
completion = client.chat.completions.create( ..., libretto=LibrettoCreateParams( template_chat=[ {"role": "ai", "content": "Hello, I'm {system_name}!"}, {"role": "user", "content": "Hi {system_name}, I'm {user_name}!"} ], ), )
To represent an array of chat messages, use the artificial role
"chat_history"
withcontent
set to the variable name in substitution format:[{"role": "chat_history", "content": "{prev_messages}"}}]
-
template_text
: The text template to record for completion requests. This is a string or a template string with{}
placeholders.For example:
completion = client.completions.create( ..., libretto=LibrettoCreateParams( template_text="Please welcome the user to {system_name}!", ), )
-
template_params
: The parameters to use for template strings. This is a dictionary of key-value pairs.For example:
completion = client.completions.create( ..., libretto=LibrettoCreateParams( template_text="Please welcome the user to {system_name}!", template_params={"system_name": "Awesome Comics Incorporated"}, ), )
-
event_id
: A unique UUID for a specific call. If not provided, one will be generated.For example:
import uuid completion = client.completions.create( ..., libretto=LibrettoCreateParams( event_id=uuid.uuid4(), ), )
-
parent_event_id
: The UUID of the parent event. All calls with the same parent id are grouped as a "Run Group".For example:
import uuid parent_id = uuid.uuid4() # First call in the run group completion = client.completions.create( ..., libretto=LibrettoCreateParams( parent_event_id=parent_id, ), ) # Another call in the same group completion = client.completions.create( ..., libretto=LibrettoCreateParams( parent_event_id=parent_id, ), )
Sending Feedback
Sometimes the answer provided by the LLM is not ideal, and your users may be able to help you find better responses. There are a few common cases:
- You might use the LLM to suggest the title of a news article, but let the user edit it. If they change the title, you can send feedback to Libretto that the answer was not ideal.
- You might provide a chatbot that answers questions, and the user can rate the answers with a thumbs up (good) or thumbs down (bad).
You can send this feedback to Libretto by calling send_feedback()
. This will
send a feedback event to Libretto about a prompt that was previously called, and
let you review this feedback in the Libretto dashboard. You can use this
feedback to develop new tests and improve your prompts.
from libretto_openai import Client
client = Client()
completion = client.completions.create(...)
# Maybe the user didn't like the answer, so ask them for a better one
better_response = ask_user_for_better_response(completion.choices[0].text)
# If the user provided a better answer, send feedback to Libretto
if better_response !== completion.choices[0].text:
# feedback key is automatically injected into OpenAI response object as an extra field
feedback_key = completion.model_extra.get("libretto_feedback_key")
client.send_feedback(
feedback_key=feedback_key,
# Better answer from the user
better_response=better_response,
# Rating of existing answer, from 0 to 1
rating=0.2,
)
Note that feedback can include either rating
, better_response
, or both.
Parameters:
rating
- a value from 0 (meaning the result was completely wrong) to 1 (meaning the result was correct)better_response
- the better response from the user
Credits
This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage
project template.
- Cookiecutter: https://github.com/audreyr/cookiecutter
audreyr/cookiecutter-pypackage
: https://github.com/audreyr/cookiecutter-pypackage
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file libretto_openai-1.0.4.tar.gz
.
File metadata
- Download URL: libretto_openai-1.0.4.tar.gz
- Upload date:
- Size: 59.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/5.0.0 CPython/3.12.3
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | a6cc952ce55666767df162b523eefe80c6ac15ebccf7bf5da1dd67d8f92339cb |
|
MD5 | 0d2e271083d51d0b349ab561db05bfc8 |
|
BLAKE2b-256 | 92ce3dacdfa84d76871c65a8cc69425f556720b93155ade7330bb627cf2ba8a8 |
File details
Details for the file libretto_openai-1.0.4-py3-none-any.whl
.
File metadata
- Download URL: libretto_openai-1.0.4-py3-none-any.whl
- Upload date:
- Size: 61.0 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/5.0.0 CPython/3.12.3
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | e4b73d91c39d9f79455bf713e453b8d701f0324a616bb928c023b5cbe73bfa31 |
|
MD5 | be87fd8d472ca61caee90a1b361a0978 |
|
BLAKE2b-256 | 557cc3ec62c6a46b4805379636f50c49ffb0bc6d23e1ab6ef212c0992a46a85c |