promptmin

Prompt minimizer for LLM evals — shrink a prompt to the smallest input that still reproduces a failure using delta debugging (ddmin).

Why?

• Fast debugging: minimal repros beat 300-line prompts
• Cheaper CI: fewer tokens, fewer moving parts
• Safer iteration: smaller diffs, clearer "what changed"
• Handles flakiness: stability modes (strict / k-of-n) and --confirm-final

Installation

This package is a Python wrapper that shells out to the promptmin Node CLI.

# 1. Install the CLI
npm install -g promptmin

# 2. Install the Python wrapper
pip install promptmin

Usage

Python API

from promptmin import minimize

result = minimize(
    prompt_path="prompts/support.md",
    config_path="promptmin.config.json",
    target="test:refund_policy_01",
    budget_runs=60
)

print(f"Minimized: {result.minimized_path}")
print(f"Report: {result.report_path}")

CLI

promptmin-py minimize \
  --prompt prompts/support.md \
  --config promptmin.config.json \
  --target test:refund_policy_01 \
  --budget-runs 60

Config example

{
  "runner": {
    "type": "openai_responses",
    "model": "gpt-4.1-mini"
  },
  "tests": [{
    "id": "refund_policy_01",
    "input": { "user": "Can I get a refund?" },
    "assert": { "type": "regex_not_match", "pattern": "competitor" }
  }]
}

Requires OPENAI_API_KEY for the openai_responses runner, or use local_command to run your own eval script.

Artifacts

After minimization, you get:

• baseline.prompt / minimized.prompt — before and after
• diff.patch — what was removed
• report.md / meta.json — summary and metadata

Options

Links

• GitHub
• Full documentation

License

MIT