Integrate and operate your products with the power of Large Language Models from a single collaboration platform. Conduct prompt engineering, experimentation, operations and monitoring across models, with full transparency on quality and costs.
Orquesta Python SDK
Contents
Installation
pip install orquesta-sdk
Creating a client instance
You can get your workspace API key from the settings section in your Orquesta
workspace. https://my.orquesta.dev/<workspace>/settings/developers
Initialize the Orquesta client with your API key:
import os
from orquesta_sdk import Orquesta, OrquestaClientOptions
api_key = os.environ.get("ORQUESTA_API_KEY", "__API_KEY__")
options = OrquestaClientOptions(
api_key=api_key,
environment="production"
)
client = Orquesta(options)
To configure connection settings when creating a client instance, use the OrquestaClientOptions class, which allows
for the adjustment of the following parameters:
OrquestaClientOptions
api_key: str - workspace API key to use for authentication.environment: Optional[str] - it is recommended, though not required, to specify the environment for the client. This ensures it is automatically added to the evaluation context.
Deployments
The Deployments API delivers text outputs, images or tool calls based on the configuration established within Orquesta for your deployments. Additionally, this API supports streaming. To ensure ease of use and minimize errors, using the code snippets from the Orquesta Admin panel is highly recommended.
Invoke a deployment
invoke()
deployment = client.deployments.invoke(
key="customer_service",
context={"environments": "production", "country": "NLD"},
inputs={"firstname": "John", "city": "New York"},
metadata={"customer_id": "Qwtqwty90281"},
)
print(deployment.choices[0].message.content)
invoke_with_stream()
deployment = client.deployments.invoke_with_stream(
key="customer_service",
context={"environments": "production", "country": "NLD"},
inputs={"firstname": "John", "city": "New York"},
metadata={"customer_id": "Qwtqwty90281"},
)
for chunk in deployment:
if chunk.is_final:
print("Stream is finished")
Adding messages as part of your request
If you are using the invoke method, you can include messages in your request to the model. The messages property
allows you to combine chat_history with the prompt configuration in Orquesta, or to directly send messages to the
model if you are managing the prompt in your code.
deployment = client.deployments.invoke(
key="Customer_service_assistant",
context={
"language": [],
"environments": []
},
metadata={
"custom-field-name": "custom-metadata-value"
},
inputs={"firstname": "John", "city": "New York"},
messages=[{
"role": "user",
"content": "A customer is asking about the latest software update features. Generate a detailed and informative response highlighting the key new features and improvements in the latest update.",
}]
)
Logging metrics to the deployment configuration
After invoking, streaming or getting the configuration of a deployment, you can use the add_metrics method to add
information to the deployment.
deployment.add_metrics(
chain_id="c4a75b53-62fa-401b-8e97-493f3d299316",
conversation_id="ee7b0c8c-eeb2-43cf-83e9-a4a49f8f13ea",
user_id="e3a202a6-461b-447c-abe2-018ba4d04cd0",
feedback={"score": 100},
metadata={
"custom": "custom_metadata",
"chain_id": "ad1231xsdaABw",
},
messages=[{
"role": "user",
"content": "A customer is asking about the latest software update features. Generate a detailed and informative response highlighting the key new features and improvements in the latest update.",
}]
)
Get deployment configuration
get_config()
config = client.deployments.get_config(
key="customer_service",
context={"environments": "production", "country": "NLD"},
inputs={"firstname": "John", "city": "New York"},
metadata={"customer_id": "Qwtqwty90281"},
)
print(config.to_dict())
Logging metrics to the deployment configuration
After invoking, streaming or getting the configuration of a deployment, you can use the add_metrics method to add
information to the deployment.
deployment.add_metrics(
chain_id="c4a75b53-62fa-401b-8e97-493f3d299316",
conversation_id="ee7b0c8c-eeb2-43cf-83e9-a4a49f8f13ea",
user_id="e3a202a6-461b-447c-abe2-018ba4d04cd0",
feedback={"score": 100},
metadata={
"custom": "custom_metadata",
"chain_id": "ad1231xsdaABw",
},
usage={
"prompt_tokens": 100,
"completion_tokens": 900,
"total_tokens": 1000,
},
performance={
"latency": 9000,
"time_to_first_token": 250,
},
)
Logging LLM responses
Whether you use the get_config or invoke, you can log the model generations to the deployment. Here are some
examples of how to do it.
Logging the completion choices the model generated for the input prompt
deployment.add_metrics(
choices=[
{
"index": 0,
"finish_reason": "assistant",
"message": {
"role": "assistant",
"content": "Dear customer: Thank you for your interest in our latest software update! We're excited to share with you the new features and improvements we've rolled out. Here's what you can look forward to in this update",
},
},
]
)
Logging the completion choices the model generated for the input prompt
You can save the images generated by the model in Orquesta. If the image format is base64 we always store it as
a png.
deployment.add_metrics(
choices=[
{
"index": 0,
"finish_reason": 'stop',
"message": {
"role": "assistant",
"url": "<image_url>"
},
},
],
)
Logging the output of the tool calls
deployment.add_metrics(
choices=[
{
"index": 0,
"message": {
"role": "assistant",
"content": None,
"tool_calls": [
{
"type": "function",
"id": "call_pDBPMMacPXOtoWhTWibW1D94",
"function": {
"name": "get_weather",
"arguments": '{"location":"San Francisco, CA"}',
},
},
],
},
"finish_reason": 'tool_calls',
}
]
)
Orquesta API
Deployments API
Class:
Methods:
-
client.deployments. get_config({ ...params }) -> `DeploymentConfig` -
client.deployments. invoke({ ...params }) -> `Deployment` -
client.deployments. invoke_with_stream({ ...params }) -> `Generator[Deployment, Any, None]`
1 maintainer
Metadata
1 maintainer
Release files for orquesta-sdk 2.0.11
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| orquesta_sdk-2.0.11.tar.gz | 11.3 kB | Details |
Release files / orquesta_sdk-2.0.11.tar.gz
| Download URL | orquesta_sdk-2.0.11.tar.gz |
|---|---|
| Size | 11.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
425f65f201a4af181ce5ac2360688ed1d4180ea77c951d009081d0fcdd1d2c93
|
|
BLAKE2b-256 checksum How to use checksums |
edd61b2b9b4c0c25e151a8d3bc78c8a2496fec9ceb7228e870cdba7d94895199
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.11.6
|
