coolprompt 2.1.0

CoolPrompt is a framework for automatic prompt creation and optimization.

Join our telegram channel to be in touch.

Practical cases

• Automatic prompt engineering for solving tasks using LLM
• (Semi-)automatic generation of markup for fine-tuning
• Formalization of response quality assessment using LLM
• Prompt for AI Agentic Pipelines
• Etc.

Core features

• Optimize prompts with our APO methods:
  • HyPER / HyPER Light
  • RE-GPS
  • RIDER
  • BRAVE
  • PromptCompressor
  • (legacy/deprecated): ReflectivePrompt, DistillPrompt
• LLM-Agnostic Choice: work with your custom llm (from open-sourced to proprietary) using supported Langchain LLMs
• Develop own custom APO method in one library
• Generate synthetic evaluation data when no input dataset is provided
• Evaluate a quality of prompts incorporating multiple metrics for both classification and generation tasks
• Evaluate costs of optimization processes by a number of tokens/calls and a price.
• Automatic task detecting for scenarios without explicit user-defined task specifications

APO methods comparison

CoolPrompt provides several automatic prompt optimization methods with different trade-offs in data requirements, runtime, expected quality, and API cost. The levels below are qualitative and task-dependent: they are intended as a quick guide for choosing a method before running a benchmark.

Compared metrics:

• Data - whether dataset is required for the method to run.
• Runtime - relative wall-clock time of one optimization run.
• Performance - expected ability to improve task quality compared with the initial prompt.
• Cost - relative compute/API cost: LLM calls, evaluation calls, token usage, and extra scoring overhead.

Quick install

• Install with pip:

pip install coolprompt

• Install with git:

git clone https://github.com/CTLab-ITMO/CoolPrompt.git
cd CoolPrompt

pip install -e .

Quick start

Set your OpenAI API key before running. The default model is gpt-4o-mini via the OpenAI API (OPENAI_API_KEY environment variable)

from coolprompt.assistant import PromptTuner

prompt_tuner = PromptTuner()

prompt_tuner.run('Write an essay about autumn')

print(prompt_tuner.final_prompt)

# You are an expert writer and seasonal observer tasked with composing a rich,
# well-structured, and vividly descriptive essay on the theme of autumn...

Run the data-driven BRAVE optimizer by selecting it as the method:

final_prompt = prompt_tuner.run(
    "Classify the sentiment of the text: {text}",
    task="classification",
    dataset=["Great product", "Very disappointing"],
    target=["positive", "negative"],
    method="brave",
    problem_description="Classify product-review sentiment.",
    max_steps=20,
    initial_budget_tokens=50_000,
)

Examples

See more examples in notebooks to familiarize yourself with our framework

About project

• The framework is developed by Computer Technologies Lab (CT-Lab) of ITMO University.
• API Reference

Contributing

• We welcome and value any contributions and collaborations, so please contact us. For new code check out CONTRIBUTING.md.

Reference

For technical details and full experimental results, please check our papers + citations inside.

@inproceedings{dragomirov2026rider,
  author = {Dragomirov, Daglar and Kulin, Nikita and Muravyov, Sergey and Makarov, Ilya and Sukhorukov, Daniil and Mozikov, Mikhail},
  title = {RIDER: Evolutionary Prompt Optimization with Adaptive Operator Selection for Software Engineering},
  booktitle = {Companion Proceedings of the 34th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering},
  series = {FSE Companion '26},
  year = {2026},
  doi = {10.1145/3803437.3807393}
}

@inproceedings{kulin2026re,
  title={RE-GPS: Reflective Evolutionary Gradient Prompting System for Large Language Models},
  author={Kulin, Nikita and Zhuravlev, Viktor and Khairullin, Artur and Muravyov, Sergey},
  booktitle={2026 39th Conference of Open Innovations Association (FRUCT)},
  pages={157--163},
  year={2026},
  organization={IEEE}
}

@INPROCEEDINGS{11239071,
  author={Kulin, Nikita and Zhuravlev, Viktor and Khairullin, Artur and Sitkina, Alena and Muravyov, Sergey},
  booktitle={2025 38th Conference of Open Innovations Association (FRUCT)}, 
  title={CoolPrompt: Automatic Prompt Optimization Framework for Large Language Models}, 
  year={2025},
  volume={},
  number={},
  pages={158-166},
  keywords={Technological innovation;Systematics;Large language models;Pipelines;Manuals;Prediction algorithms;Libraries;Prompt engineering;Optimization;Synthetic data},
  doi={10.23919/FRUCT67853.2025.11239071}
}

@misc{zhuravlev2025reflectivepromptreflectiveevolutionautoprompting,
      title={ReflectivePrompt: Reflective evolution in autoprompting algorithms}, 
      author={Viktor N. Zhuravlev and Artur R. Khairullin and Ernest A. Dyagin and Alena N. Sitkina and Nikita I. Kulin},
      year={2025},
      eprint={2508.18870},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2508.18870}, 
}

@misc{dyagin2025automaticpromptoptimizationprompt,
      title={Automatic Prompt Optimization with Prompt Distillation},
      author={Ernest A. Dyagin and Nikita I. Kulin and Artur R. Khairullin and Viktor N. Zhuravlev and Alena N. Sitkina},
      year={2025},
      eprint={2508.18992},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2508.18992}, 
}