prompt-injection-bench 0.1.1

Package to test Prompt Injection Against OpenAI's ChatGPT

ChatGPT-4 for Jailbreak Dataset Analysis

This repository contains Python code to analyze the Hugging Face Jailbreak dataset using OpenAI's ChatGPT-4 model. The code sends prompts from the dataset to ChatGPT-4 to evaluate responses and detect potential prompt injection attacks.

Anxious to check the results?

As of January 10th 2024, GPT-4 detect all jailbreaks on this dataset. Moreover, the word jailbreak is used very loosely, some jailbreak prompts are of no consequence.

Requirements

To run this code, you need the following:

• Python 3
• OpenAI Python library
• Hugging Face datasets library
• A valid OpenAI API key

Install the required libraries using pip3 and the provided requirements.txt file:

pip3 install -r requirements.txt

Setup

1. Clone this repository:

git clone https://github.com/BenderScript/PromptInjectionBench.git
cd PromptInjectionBench

2. Create a .env file in the project directory and add your OpenAI API key:

OPENAI_API_KEY=your_api_key_here

Running the Analysis

To analyze the Jailbreak dataset with ChatGPT-4, execute the following Python script:

python3 main.py

This script loads the dataset, iterates through prompts, sends them to ChatGPT-4, and detects potential injection attacks in the generated responses.

Code Structure

• main.py: The main script that loads the dataset, defines functions to interact with ChatGPT-4, and performs the analysis.

Skipping "Benign" Prompts

In the interest of time, the code skips prompts labeled as "benign." This helps focus the analysis on potentially harmful prompts where injection attacks might occur.

Tabulating Attacks vs. Detected Attacks

The script tabulates the total number of generated texts and the number of texts where potential injection attacks are detected. It counts these attacks based on the presence of specific keywords such as "I'm sorry," "apologize," or "developed by OpenAI" in the generated text.

Results

The script will print information about each prompt, including its type, the generated text from ChatGPT-4, and whether an injection attack was detected based on the presence of specific keywords.

License

This code is provided under the Apache License 2.0. Feel free to use and modify it as needed.

---

This analysis is provided as a reference and demonstration of using OpenAI's ChatGPT-4 model for evaluating prompt injection attacks in text datasets.

For more information about OpenAI's GPT-4 model and the Hugging Face Jailbreak dataset, please refer to the official documentation and sources:

• OpenAI GPT-4
• Hugging Face Jailbreak Dataset

These explanations added to the README.md should help users understand why "benign" prompts are skipped and how the code tabulates attacks vs. detected attacks.