coolprompt 1.0.1

Framework for automative prompting creation and optimization

English | Русский

CoolPrompt is a framework for automative prompting creation and optimization.

Practical cases

• Automatic prompt engineering for solving tasks using LLM
• (Semi-)automatic generation of markup for fine-tuning
• Formalization of response quality assessment using LLM
• Prompt tuning for agent systems

Quick install

• Install with pip:

pip install coolprompt

• Install with git:

git clone https://github.com/CTLab-ITMO/CoolPrompt.git

pip install -r requirements.txt

Quick start

Import and initialize PromptTuner

from coolprompt.assistant import PromptTuner

• with default LLM

prompt_tuner = PromptTuner()

• or customize your own LLM using supported Langchain LLMs
• List of available LLMs: https://python.langchain.com/docs/integrations/llms/

from langchain_community.llms import VLLM

my_model = VLLM(
    model="Qwen/Qwen2.5-Coder-32B-Instruct",
    trust_remote_code=True,
    dtype='bfloat16',
)

prompt_tuner = PromptTuner(model=my_model)

Running PromptTuner

• Run PromptTuner instance with initial prompt

# Define an initial prompt
prompt = "Make a summarization of 2+2"

# Run a prompt optimisation
new_prompt = tuner.run(start_prompt=prompt)

# Get your new prompt
print(new_prompt)

• including a dataset for prompt optimization and evaluation. A provided dataset will be split by trainset and testset.

sst2 = load_dataset("sst2")
class_dataset = sst2['train']['sentence']
class_targets = sst2['train']['label']

tuner.run(
    start_prompt=class_start_prompt,
    task="classification",
    dataset=class_dataset,
    target=class_targets,
    metric="accuracy"
)

• to get a final prompt and prompt metrics

print("Final prompt:", tuner.final_prompt)
print("Start prompt metric:", tuner.init_metric)
print("Final prompt metric:", tuner.final_metric)

• This also works for generation tasks

More about project

• Explore the variety of autoprompting methods with PromptTuner: CoolPrompt currently support HyPE, DistillPrompt, ReflectivePrompt. You can choose method via corresponding argument method in tuner.run
• See more examples in notebooks to familiarize yourself with our framework