promptcraft 0.3.0

PromptCraft: A Prompt Perturbation Toolkit for Prompt Robustness Analysis

PromptCraft

A Prompt Perturbation Toolkit for Prompt Robustness Analysis

Table of Contents

• Installation
• Character Editing
  • Character Replacement
  • Character Deletion
  • Character Insertion
  • Character Swap
  • Keyboard Typos
  • Optical Character Recognition (OCR)
• Word Manipulation
  • Synonym Replacement
  • Word Insertion
  • Word Swap
  • Word Deletion
  • Insert Punctuation
  • Word Split
• Sentence Paraphrasing
  • Back Translation based on 🤗 Hugging Face MarianMTModel
  • Back Translation based on Google Translator
  • Paraphrasing
  • Formal Style
  • Casual Style
  • Passive Style
  • Active Style
• Parallel Processing
• Structure of the Code
• Acknowledgement

Installation

pip install promptcraft

Character Editing

from promptcraft import character

sentence = "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May."
level = 0.25  # Number of characters that will be edited

character_tool = character.CharacterPerturb(sentence=sentence, level=level)

Character Replacement

char_replace = character_tool.character_replacement()

Character Deletion

char_delete = character_tool.character_deletion()

Character Insertion

char_insert = character_tool.character_insertion()

Character Swap

char_swap = character_tool.character_swap()

Keyboard Typos

char_keyboard = character_tool.keyboard_typos()

Optical Character Recognition

char_ocr = character_tool.optical_character_recognition()

Word Manipulation

from promptcraft import word

sentence = "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May."
level = 0.25  # Number of words that will be manipulated
word_tool = word.WordPerturb(sentence=sentence, level=level)

Synonym Replacement

word_synonym = word_tool.synonym_replacement()

Word Insertion

word_insert = word_tool.word_insertion()

Word Swap

word_swap = word_tool.word_swap()

Word Deletion

word_delete = word_tool.word_deletion()

Insert Punctuation

word_punctuation = word_tool.insert_punctuation()

Word Split

word_split = word_tool.word_split()

Sentence Paraphrasing

from promptcraft import sentence

sen = "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May."
sentence_tool = sentence.SentencePerturb(sentence=sen)

Back Translation by Hugging Face

back_trans_hf = sentence_tool.back_translation_hugging_face()

Back Translation by Google Translator

back_trans_google = sentence_tool.back_translation_google()

Paraphrasing

sen_paraphrase = sentence_tool.paraphrase()

Formal Style

sen_formal = sentence_tool.formal()

Casual Style

sen_casual = sentence_tool.casual()

Passive Style

sen_passive = sentence_tool.passive()

Active Style

sen_active = sentence_tool.active()

Parallel Processing

Since all the methods are executed on the CPU, they can be performed in parallel using the multiprocessing package.

Structure of the Code

At the root of the project, you will see:

.
├── LICENSE
├── README.md
├── promptcraft
│   ├── __init__.py
│   ├── character.py
│   ├── parrot.py
│   ├── sentence.py
│   ├── styleformer.py
│   └── word.py
└── setup.py

Acknowledgement

This work was finished during my 2023 fall semester research rotation at the Dependable Computing Laboratory, Department of Electrical and Computer Engineering, Boston University, under the supervision of Prof. Wenchao Li.