orquesta-sdk 2.0.0rc1

_LLM Operations and Integration Platform_

LLM Operations and Integration Platform

Orquesta Python SDK

Contents

• Installation
• Create a client instance
• Usage
• Reference

Installation

pip install orquesta-sdk

Creating a client instance

You can get your workspace API key from the settings section in your Orquesta workspace. https://my.orquesta.dev/<workspace>/settings/developers

Initialize the Orquesta client with your API key:

import os

from orquesta_sdk import Orquesta, OrquestaClientOptions

api_key = os.environ.get("ORQUESTA_API_KEY", "__API_KEY__")

options = OrquestaClientOptions(
    api_key=api_key,
    environment="production"
)

client = Orquesta(options)

To configure connection settings when creating a client instance, use the OrquestaClientOptions class, which allows for the adjustment of the following parameters:

OrquestaClientOptions

• api_key: str - workspace API key to use for authentication.
• environment: Optional[str] - it is recommended, though not required, to specify the environment for the client. This ensures it is automatically added to the evaluation context.

Deployments

The Deployments API delivers text outputs, images or tool calls based on the configuration established within Orquesta for your deployments. Additionally, this API supports streaming. To ensure ease of use and minimize errors, using the code snippets from the Orquesta Admin panel is highly recommended.

Invoke a deployment

invoke()

deployment = client.deployments.invoke(
     key="customer_service",
    context={"environments": "production", "country": "NLD"},
    inputs={"firstname": "John", "city": "New York"},
    metadata={"customer_id": "Qwtqwty90281"},
)

print(deployment.choices[0].message.content)

invoke_with_stream()

deployment = client.deployments.invoke_with_stream(
    key="customer_service",
    context={ "environments": "production", "country": "NLD" },
    inputs={ "firstname": "John", "city": "New York" },
    metadata={ "customer_id": "Qwtqwty90281" },
)

for chunk in deployment:
    if chunk.is_final:
        print("Stream is finished")

Logging metrics to the deployment configuration

After invoking, streaming or getting the configuration of a deployment, you can use the add_metrics method to add information to the deployment.

deployment.add_metrics(
    chain_id="c4a75b53-62fa-401b-8e97-493f3d299316",
    conversation_id="ee7b0c8c-eeb2-43cf-83e9-a4a49f8f13ea",
    user_id="e3a202a6-461b-447c-abe2-018ba4d04cd0",
    feedback={"score": 100},
    metadata={
        "custom": "custom_metadata",
        "chain_id": "ad1231xsdaABw",
    }
)

Get deployment configuration

get_config()

config = client.deployments.get_config(
    key="customer_service",
    context={ "environments": "production", "country": "NLD" },
    inputs={ "firstname": "John", "city": "New York" },
    metadata={ "customer_id": "Qwtqwty90281" },
)

print(config.to_dict())

Logging metrics to the deployment configuration

After invoking, streaming or getting the configuration of a deployment, you can use the add_metrics method to add information to the deployment.

deployment.add_metrics(
    chain_id="c4a75b53-62fa-401b-8e97-493f3d299316",
    conversation_id="ee7b0c8c-eeb2-43cf-83e9-a4a49f8f13ea",
    user_id="e3a202a6-461b-447c-abe2-018ba4d04cd0",
    feedback={"score": 100},
    metadata={
        "custom": "custom_metadata",
        "chain_id": "ad1231xsdaABw",
    },
    usage={
        "prompt_tokens": 100,
        "completion_tokens": 900,
        "total_tokens": 1000,
    },
    performance={
        "latency": 9000,
        "time_to_first_token": 250,
    },
)

Orquesta API

Deployments API

Class:

• Deployments
• Deployment
• DeploymentConfig

Methods:

• client.deployments.get_config({ ...params }) -> DeploymentConfig
• client.deployments.invoke({ ...params }) -> Deployment
• client.deployments.invoke_with_stream({ ...params }) -> Generator[Deployment, Any, None]

Examples

You can find more examples in the notebooks folder. In there we cover how to use the SDK in different scenarios to handle text models, image models and tool calling.