LaunchDarkly AI SDK — optimization

This package provides helpers for running iterative AI prompt optimization workflows from within LaunchDarkly SDK-based applications. It drives the optimization loop — generating candidate variations, evaluating them with judges, and optionally committing winners back to LaunchDarkly — while delegating all LLM calls to your own handler functions.

Requirements

• Python >=3.9
• A configured LaunchDarkly server-side SDK client
• The LaunchDarkly AI package (launchdarkly-server-sdk-ai>=0.16.0) — pulled in automatically as a dependency
• LAUNCHDARKLY_API_KEY environment variable — required only when using auto_commit=True or optimize_from_config. Not needed for basic optimize_from_options runs without auto-commit.

Installation

pip install launchdarkly-ai-optimizer

Quick Start

Basic optimization (optimize_from_options)

No LAUNCHDARKLY_API_KEY required unless auto_commit=True.

import ldclient
from ldai import LDAIClient
from ldai_optimizer import (
    OptimizationClient,
    OptimizationJudge,
    OptimizationOptions,
    OptimizationResponse,
    LLMCallConfig,
    LLMCallContext,
)

ldclient.set_config(ldclient.Config("sdk-your-sdk-key"))
ld = LDAIClient(ldclient.get())
client = OptimizationClient(ld)

def handle_llm_call(
    run_id: str,
    config: LLMCallConfig,
    context: LLMCallContext,
    is_evaluation: bool,
) -> OptimizationResponse:
    # config.model, config.instructions, config.key are available
    # context.user_input, context.current_variables are available
    response = your_llm_client.chat(
        model=config.model.name if config.model else "gpt-4o",
        system=config.instructions,
        user=context.user_input or "",
    )
    return OptimizationResponse(completion=response.text)

result = await client.optimize_from_options(
    OptimizationOptions(
        agent_key="my-agent",
        handle_agent_call=handle_llm_call,
        judge_model="gpt-4o-mini",
        judges={
            "quality": OptimizationJudge(
                threshold=1.0,
                acceptance_statement="The response is accurate and concise.",
            )
        },
        model_choices=["gpt-4o", "gpt-4o-mini"],
        variable_choices=[{"user_id": "user-123"}],
        user_input_choices=["What is my account balance?"],
    )
)

Ground truth optimization

from ldai_optimizer import GroundTruthOptimizationOptions, GroundTruthSample

result = await client.optimize_from_options(
    GroundTruthOptimizationOptions(
        agent_key="my-agent",
        handle_agent_call=handle_llm_call,
        judge_model="gpt-4o-mini",
        judges={
            "accuracy": OptimizationJudge(
                threshold=1.0,
                acceptance_statement="The response matches the expected answer.",
            )
        },
        model_choices=["gpt-4o", "gpt-4o-mini"],
        ground_truth_responses=[
            GroundTruthSample(
                user_input="What is 2+2?",
                ground_truth_response="4",
            )
        ],
    )
)

Config-driven optimization (optimize_from_config)

Requires LAUNCHDARKLY_API_KEY.

from ldai_optimizer import OptimizationFromConfigOptions

result = await client.optimize_from_config(
    OptimizationFromConfigOptions(
        config_key="my-optimization-config",
        project_key="my-project",
        handle_agent_call=handle_llm_call,
        auto_commit=True,
    )
)

License

Apache-2.0