Promptic Python SDK

Python SDK and CLI for the Promptic platform — tracing, prompt optimization, and experiment management.

Installation

pip install promptic-sdk

Optional LLM instrumentation

Install extras to auto-instrument specific providers or agent frameworks:

# LLM providers
pip install promptic-sdk[openai]         # OpenAI
pip install promptic-sdk[anthropic]      # Anthropic
pip install promptic-sdk[bedrock]        # AWS Bedrock
pip install promptic-sdk[vertexai]       # Google Vertex AI
pip install promptic-sdk[mistralai]      # Mistral

# Agent frameworks
pip install promptic-sdk[langchain]      # LangChain / LangGraph / create_agent / deepagents
pip install promptic-sdk[openai-agents]  # OpenAI Agents SDK
pip install promptic-sdk[claude-agent]   # Claude Agent SDK

pip install promptic-sdk[all]            # Everything above

Pydantic AI ships its own OpenTelemetry emitter — enable it with Agent(..., instrument=True); no extras needed.

Quick start

1. Authenticate

Log in via browser (recommended for local development):

promptic login

This opens your browser for authentication, then auto-selects your AI Application. Credentials are saved to ~/.promptic/config.toml.

For CI/CD or headless environments, use an API key instead:

promptic configure
# or set the environment variable:
export PROMPTIC_API_KEY="ptc_..."

2. Send traces

import promptic_sdk
from openai import OpenAI

# Initialize tracing (auto-instruments installed LLM libraries)
promptic_sdk.init()

client = OpenAI()

# Tag traces with an AI Component name
with promptic_sdk.ai_component("customer-support-agent"):
    response = client.chat.completions.create(
        model="gpt-4.1-nano",
        messages=[{"role": "user", "content": "Hello!"}],
    )

3. Use the API client

from promptic_sdk import PrompticClient

with PrompticClient() as client:
    # List traces
    traces = client.list_traces(limit=10)

    # Get AI Application info
    ai_application = client.get_ai_application()

    # Manage experiments
    experiment = client.create_experiment(
        ai_component_id="comp_...",
        target_model="gpt-4.1-nano",
        task_type="classification",
        initial_prompt="Classify the following text.",
    )

    # Deploy the best prompt
    client.deploy(component_id="comp_...", experiment_id="exp_...")

    # Fetch a deployed prompt at runtime
    prompt = client.get_deployed_prompt("comp_...")

Tool-selection experiments

Prompt experiments use create_experiment() with classification, textGeneration, or structuredOutput. Tool selection requires additional configuration and uses a typed, all-or-nothing workflow:

with PrompticClient() as client:
    experiment = client.create_tool_selection_experiment(
        "comp_...",
        tools=[{"name": "get_weather", "description": "Get weather for a city"}],
        test_cases=[{"query": "Weather in Berlin?", "expected_tool": "get_weather"}],
        target_model="gpt-4.1-nano",
        tool_source="manual",
        optimize_system_prompt=True,
    )
    client.start_experiment(experiment["id"])

The experiment dataset is managed automatically and deleted with the experiment. Use create_dataset() for a named, reusable dataset shared by evaluations or other workflows. Each iteration returned by list_iterations(), get_iteration(), or get_best_iteration() may include toolDescriptions (the optimized description keyed by tool name) and selectionSystemPrompt when system-prompt optimization is enabled.

Tracing

promptic_sdk.init() sets up OpenTelemetry to export spans to the Promptic platform.

Auto-detected instrumentors: OpenAI, Anthropic, Google Generative AI, Vertex AI, Bedrock, Mistral, Cohere, LangChain (with LangGraph / deepagents), OpenAI Agents SDK, Claude Agent SDK. All emit the official OpenTelemetry GenAI semantic conventions (gen_ai.*), so traces work uniformly across frameworks.

Using other OpenTelemetry instrumentors

Since Promptic uses standard OpenTelemetry under the hood, you can add any OTel-compatible instrumentor alongside the auto-detected ones. Just call promptic_sdk.init() first, then instrument manually:

import promptic_sdk
from opentelemetry.instrumentation.requests import RequestsInstrumentor
from opentelemetry.instrumentation.sqlalchemy import SQLAlchemyInstrumentor

promptic_sdk.init()

# Add any OpenTelemetry instrumentor — spans will be exported to Promptic
RequestsInstrumentor().instrument()
SQLAlchemyInstrumentor().instrument(engine=engine)

This works with any package from the opentelemetry-python-contrib ecosystem (HTTP clients, databases, web frameworks, etc.). All spans are exported to the Promptic platform as long as init() has been called.

AI Components

Use ai_component() to tag spans with a component name. The platform links traces to the matching AI Component in your AI Application:

with promptic_sdk.ai_component("my-component"):
    # All LLM calls here are tagged
    ...

To add traces to a dataset, pass the dataset's immutable UUID. Datasets are created explicitly in the dashboard, CLI, or API; tracing never creates one from a display name.

with promptic_sdk.ai_component(
    "my-component",
    dataset_id="550e8400-e29b-41d4-a716-446655440000",
    run="v1-baseline",
):
    agent.run(test_input)

Invalid dataset IDs and run contexts without a dataset ID fail immediately, before spans are created. The SDK emits the promptic.dataset.id OpenTelemetry attribute for server-side linkage.

Tracing workflows with custom spans

Most users don't need this. With the right [extras] installed, auto-instrumentation already creates spans for every LLM and tool call. Reach for custom spans only when you have meaningful non-LLM workflow logic (retrieval, normalization, business rules, control flow) you want represented in the trace.

When you do need it, wrap your workflow stages in custom OpenTelemetry spans. Auto-instrumented LLM and tool spans automatically nest under whichever custom span is active.

import json
import promptic_sdk
from opentelemetry import trace

promptic_sdk.init()
tracer = trace.get_tracer(__name__)

with promptic_sdk.ai_component("support-agent"):
    with tracer.start_as_current_span("answer_question") as root:
        root.set_attribute("traceloop.span.kind", "workflow")
        root.set_attribute("traceloop.entity.input", json.dumps(user_input))

        with tracer.start_as_current_span("retrieve_context") as span:
            span.set_attribute("traceloop.span.kind", "task")
            span.set_attribute("traceloop.entity.input", json.dumps(query))
            context = retrieve(query)
            span.set_attribute("traceloop.entity.output", json.dumps(context))

        with tracer.start_as_current_span("generate_answer") as span:
            span.set_attribute("traceloop.span.kind", "task")
            # The auto-instrumented LLM call nests under this task span
            answer = llm_call(context)

        root.set_attribute("traceloop.entity.output", json.dumps(answer))

Span attribute conventions:

• traceloop.span.kind="workflow" — the top-level run
• traceloop.span.kind="task" — an internal pipeline stage
• traceloop.entity.input / traceloop.entity.output — JSON-serialized stage payloads, surfaced in the Promptic UI

Promptic automatically offloads inline base64 media and large file-like content into trace artifacts, then keeps lightweight promptic-artifact://... references in span attributes. Local filesystem paths are not read automatically; attach local files explicitly when you want the bytes available from the trace:

file_ref = promptic_sdk.artifact("/tmp/report.pdf")
span.set_attribute("retrieval.input_file", file_ref.ref)

The artifact's name is stored on the record and used as the default download filename. For local files it defaults to the file's base name; pass name= to override it (also available for bytes and text content), and read it back from file_ref.name:

file_ref = promptic_sdk.artifact(pdf_bytes, name="quarterly-report.pdf")

For huge collections, still log a small preview plus a count rather than the full object:

span.set_attribute(
    "traceloop.entity.output",
    json.dumps({
        "items": items[:5],
        "item_count": len(items),
        "additional_item_count": max(len(items) - 5, 0),
    }),
)

See the Tracing guide for the full pattern.

API client

Both a sync (PrompticClient) and async (AsyncPrompticClient) client are available. They share the same method signatures and return types.

from promptic_sdk import PrompticClient

with PrompticClient() as client:
    traces = client.list_traces(limit=10)

from promptic_sdk import AsyncPrompticClient

async with AsyncPrompticClient() as client:
    traces = await client.list_traces(limit=10)

Both clients provide typed methods for the full Promptic REST API:

The client reads PROMPTIC_API_KEY and PROMPTIC_ENDPOINT from the environment, or accepts them as constructor arguments.

CLI

The promptic CLI mirrors the API client and supports both human-readable tables and --json output.

promptic [command] [subcommand] [options]

Commands

All list commands support --json for machine-readable output.

Configuration

The SDK and CLI resolve configuration in this order:

1. Explicit arguments (api_key=, endpoint=)
2. Environment variables (PROMPTIC_API_KEY, PROMPTIC_ENDPOINT)
3. Config file (~/.promptic/config.toml, written by promptic login or promptic configure)

Development

Requires Python 3.11+ and uv.

# Install dependencies
uv sync

# Run tests
uv run pytest

# Lint
uv run ruff check .
uv run ruff format .

License

MIT — see LICENSE for details.