Defining and constructing production-grade LLM prompts via rich structured templates.
Project description
Project Description
Prompt Poet (PP)
Defining and constructing production-grade LLM prompts via rich structured templates.
Usage
Below is an example of the example_template.yml.j2 file structure. This template demonstrates how to include variable interpolation within a YAML file.
Filename: example_template.yml.j2
- name: first part
raw_string: string with variable interpolation: {{ var }}.
from prompt_poet import Prompt
prompt = Prompt(template_name="example_template.yml.j2", template_data={"var": "foobar"})
print(promp.string)
>>> 'string with variable interpolation: foobar.'
Or use an example template provided:
from prompt_poet import Prompt, CAIMessage
template_data = {
"timestamp": "2024 06 24",
"username": "Jeff",
"character": {
"title": "The title",
"description": "The description",
"definition": "The definition\nWith multiple lines\nIn the definition",
"participant__name": "Alice",
},
"persona_definition": "The persona definition",
"cai_messages": [
CAIMessage(author="Alice", text="The first message"),
CAIMessage(author="Jeff", text="The second message"),
CAIMessage(author="Alice", text="The third message", is_pinned=True),
CAIMessage(author="Jeff", text="The fourth message"),
],
"reply_prompt": "Alice:"
}
prompt = Prompt(template_name="cai.yml.j2", from_examples=True, template_data=template_data)
print(prompt.string)
>>> '<|beginningofdialog|>2024 06 24 Alice: The title - The description<|endofmessage|><|beginningofmessage|>narrator: The definition<|endofmessage|><|beginningofmessage|>narrator: With multiple lines<|endofmessage|><|beginningofmessage|>narrator: In the definition<|endofmessage|><|beginningofmessage|>Jeff: The persona definition<|endofmessage|><|beginningofmessage|>Alice: The third message<|endofmessage|><|beginningofmessage|>Alice: The first message<|endofmessage|><|beginningofmessage|>Jeff: The second message<|endofmessage|><|beginningofmessage|>Alice: The third message<|endofmessage|><|beginningofmessage|>Jeff: The fourth message<|endofmessage|><|beginningofmessage|>Alice:'
Goals & Requirements
- Centralized: By centralizing the prompt construction into one place, Prompt Poet (PP) aims to simplify the prompt construction mechanics.
- Extensible: Make it as easy as possible to create new prompts for new use cases.
- Experimentation: Must be easy to experiment with wholly different prompt formats and content.
Fundamentally, Prompt Poet (PP) is split into two layers -- the Templating Layer and the Logical Layer. We aim to keep a clear separation between these layers such that the Templating Layer exclusively handles the representation of the prompt and the Logical Layer handles the mechanics of constructing the prompt.
Templating Layer
Templates
Prompt templates are expressive and human-readable files that define the prompt structure, data placements, and formatting of the final prompt. The templating engine aims to strike a balance between being readable and explicit with no magic under-the-hood. As such, we have chosen to use a combination of YAML and Jinja syntax to represent prompt templates -- a common templating language for DevOps tools like Ansible.
Fundamentally, prompt templates are YAML files. Once the jinja syntax is fully rendered they contain a repeating sequence of prompt parts. Each part contains the following fields:
name
: a unique name given to the prompt part used for readability and sometimes used functionally such as for truncation.raw_string
: the string payload representing the content to be tokenized for this prompt part.truncation_priority
: the priority given to the prompt part during truncation.
We construct the final prompt by concatenating the raw_string
of these parts together to form the final prompt and do a best effort attempt at following the truncation policy implicit in the truncation_priority
field. In the future we may support additional fields. We use Jinja syntax to express arbitrary complexity in the templates such as control flow and function calls.
Alternatives considered
- Langchain: PromptTemplate, AIMessage, SystemMessage, and HumanMessage abstractions. Basically just f-strings wrapped in a Python class, not expressive enough as it does not handle control flow well.
- LMQL: Not very readable, non-trivial to reason about what the final interpolated prompt would look like.
- Raw Python f-strings: Better readability but not very expressive.
- Jinja: Probably the best standalone bet found so far but leaves several things to be desired. See an example here.
- YAML: Could also work by rolling our own basic interpreter. See an example here.
- Several OSS “prompt management” solutions: Pezzo, Agenta, PromptHub (paid), Langflow. These all miss the mark in terms of extensibility of the core templating language and infrastructure and focus on using external APIs rather than needing to truncate and tokenize, which is crucial for us as we host our own models.
Template Registry:
The Template Registry is a central repository where all prompt templates are stored. It serves as a single source of truth for prompt definitions, ensuring consistency and ease of access. The registry allows users to:
- Browse existing templates.
- Add new templates.
- Update existing templates.
- Version control templates to track changes over time.
- Validate templates to ensure they meet required standards.
Logical Layer
The Logical Layer contains the necessary logic for rendering templates and performing tokenization and truncation. It handles the dynamic aspects of prompt construction, ensuring that the final prompts adhere to specified length constraints and are correctly formatted.
1. Rendering:
The rendering process takes a template and fills in the necessary data to produce a final prompt. This involves:
- Data binding: Inserting data into the template.
- Evaluation: Processing any conditional logic or loops defined in the template.
- Formatting: Ensuring the final output is correctly structured.
2. Tokenization
Tokenization is the process of breaking down the final prompt into tokens that the LLM can understand. This involves:
- Identifying token boundaries: Ensuring the prompt is split into meaningful units.
- Handling special characters: Properly encoding or escaping characters that have special meaning in the model's tokenization scheme.
3. Truncation
Truncation ensures the prompt fits within the model's maximum token_limit
.
This involves:
- Prioritizing parts: Using the
truncation_priority
field to determine which parts of the prompt can be truncated if necessary. - Smart truncation: Preserving the most important parts of the prompt while removing less critical ones.
4. Validation
Validation ensures the final prompt adheres to all specified constraints and formatting rules. This involves:
- Schema validation: Ensuring the prompt meets the defined schema requirements.
- Token limit validation: Ensuring the prompt does not exceed the model's maximum token limit.
By adhering to these principles and design choices, Prompt Poet (PP) aims to provide a robust, flexible, and easy-to-use system for constructing high-quality LLM prompts.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for prompt_poet-0.0.13-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 583503be98ff3e6e6bfe7d81ea8510dfc18606d2fa1523783f540d2ab0e03bfb |
|
MD5 | b5176c0e3a27bbb8d384c96285de978d |
|
BLAKE2b-256 | 8426777415313725db56dfaf518e07b9e1b2b302327929f4274eb9751501585f |