Prompty — Python Runtime

Prompty is a markdown file format (.prompty) for LLM prompts. The Python runtime loads, renders, parses, and executes these files. Every .prompty file becomes a typed Prompty object.

Installation

# Core + Jinja2 renderer + OpenAI provider
uv pip install "prompty[jinja2,openai]"

# Core + Microsoft Foundry provider
uv pip install "prompty[jinja2,foundry]"

# Core + Anthropic provider
uv pip install "prompty[jinja2,anthropic]"

# Everything (all renderers, providers, OpenTelemetry)
uv pip install "prompty[all]"

Extras

Quick Start

1. Write a .prompty File

---
name: greeting
model:
  id: gpt-4o-mini
  provider: openai
  connection:
    kind: key
    apiKey: ${env:OPENAI_API_KEY}
inputs:
  - name: name
    kind: string
    default: World
template:
  format:
    kind: jinja2
  parser:
    kind: prompty
---
system:
You are a friendly assistant.

user:
Say hello to {{name}}.

2. Run It

import prompty

# One-shot: load + prepare + run
result = prompty.invoke("greeting.prompty", inputs={"name": "Jane"})
print(result)

# Step-by-step
agent = prompty.load("greeting.prompty")
messages = prompty.prepare(agent, inputs={"name": "Jane"})
result = prompty.run(agent, messages)

3. Async

result = await prompty.invoke_async("greeting.prompty", inputs={"name": "Jane"})

API Reference

Core Functions

All functions have _async variants (e.g., invoke_async, run_async, turn_async).

Agent Turns (Tool Calling)

def get_weather(location: str) -> str:
    return f"72°F and sunny in {location}"


result = prompty.turn(
    "my-agent.prompty",
    inputs={"question": "Weather in Seattle?"},
    tools={"get_weather": get_weather},
    max_iterations=10,  # default
)

The agent loop: calls the LLM, if it returns tool_calls, executes the matching functions, sends results back, repeats until the model returns a normal response or max_iterations is exceeded.

Error handling:

• Bad JSON in tool arguments → error sent back to model for self-correction
• Tool exception → error string sent back to model
• Missing tool → error message sent back (no crash)
• Max iterations → ValueError raised

Connection Registry

For pre-configured SDK clients or token-based auth:

# .prompty file — reference a named connection
model:
  id: gpt-4o
  provider: foundry
  connection:
    kind: reference
    name: my-foundry

from openai import AzureOpenAI
from azure.identity import (
    DefaultAzureCredential,
    get_bearer_token_provider,
)

client = AzureOpenAI(
    azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
    azure_ad_token_provider=get_bearer_token_provider(
        DefaultAzureCredential(),
        "https://cognitiveservices.azure.com/.default",
    ),
)
prompty.register_connection("my-foundry", client=client)

# Now run — executor resolves the client by name
result = prompty.invoke("my-prompt.prompty", inputs={...})

Structured Output

Define outputs in frontmatter to get JSON-parsed results:

outputs:
  - name: city
    kind: string
    description: The city name
  - name: temperature
    kind: integer
    description: Temperature in Fahrenheit

The executor sends this as OpenAI's response_format with json_schema, and the processor automatically JSON-parses the result.

Streaming

agent = prompty.load("chat.prompty")
messages = prompty.prepare(agent, inputs={"question": "Tell me a story"})

# Set stream option
agent.model.options.additionalProperties = {
    "stream": True,
}
response = prompty.run(agent, messages, raw=True)

# response is a PromptyStream — iterate for chunks
for chunk in prompty.process(agent, response):
    print(chunk, end="", flush=True)

Streaming handles tool call deltas (accumulated across chunks), refusal detection, and empty heartbeat chunks.

Tracing

from prompty import Tracer, PromptyTracer, trace

# Register a tracer
Tracer.add("console", prompty.console_tracer)
Tracer.add("json", PromptyTracer("./traces").tracer)

# All pipeline functions automatically emit traces
result = prompty.invoke("my-prompt.prompty", inputs={...})


# Custom functions
@trace
def my_function(): ...

OpenTelemetry integration:

from prompty.tracing.otel import otel_tracer

Tracer.add("otel", otel_tracer())

.prompty File Format

Structure

---
# YAML frontmatter (Prompty schema)
---
# Markdown body (becomes `instructions`)
# Uses role markers and template syntax

Role Markers

system:
You are a helpful assistant.

user:
{{question}}

assistant:
I'd be happy to help.

Thread Inputs

Conversation history is passed via a kind: thread input property:

inputs:
  - name: conversation
    kind: thread

Place {{conversation}} in the template where thread messages should be injected:

system:
You are a helpful assistant.
{{conversation}}
user:
{{question}}

The caller passes a list of messages as the thread input value.

Variable References

${file:...} references are scoped to the containing .prompty file's directory by default. Pass allowed_file_roots to load() / load_async() only when the host application explicitly trusts additional shared directories.

Endpoint Trust

A .prompty file can't send ambient credentials (DefaultAzureCredential tokens, OPENAI_API_KEY and similar env keys) to an endpoint it picked itself. Unless the connection uses a literal apiKey (kind: key), a literal endpoint/tokenUrl must be trusted, or load() raises PromptyLoadError("untrusted_endpoint").

• Built-in trusted origins: api.openai.com, api.anthropic.com, *.openai.azure.com, *.services.ai.azure.com, *.cognitiveservices.azure.com, login.microsoftonline.com (all https).
• Supply the endpoint from the host: endpoint: ${env:MY_ENDPOINT}.
• Or trust more origins: load(path, trusted_endpoints=["http://localhost:11434"]) or PROMPTY_TRUSTED_ENDPOINTS=http://localhost:11434,https://*.contoso.com.

Connection Kinds

Legacy Format Support

Prompty v1 .prompty files are automatically migrated with deprecation warnings. Key changes:

• model.configuration → model.connection
• model.parameters → model.options
• inputs (dict format) → inputs (list of Property)
• older input schema fields → inputs
• older output schema fields → outputs
• template: jinja2 → structured template with format and parser

Development

# Install for development
uv pip install -e ".[dev,all]"

# Unit tests (excludes integration by default)
uv run pytest tests/ -q

# Lint
uv run ruff check .
uv run ruff format --check .

# Integration tests (requires API keys in .env)
uv run pytest tests/integration/ -v -o "addopts="

License

See LICENSE in the repository root.