prompt-injection-bench 0.1.6

Package to test Prompt Injection Against OpenAI's ChatGPT

ChatGPT-4 and Gemini Pro for Jailbreak Dataset Analysis (CyberPunk mode)

This repository contains Python code to analyze the Hugging Face Jailbreak dataset using OpenAI's ChatGPT-4 model. The code sends prompts from the dataset to ChatGPT-4 to evaluate responses and detect potential prompt injection attacks.

Anxious to check the results?

As of January 10th 2024, GPT-4 detect all jailbreaks on this dataset. Moreover, the word jailbreak is used very loosely, some jailbreak prompts are of no consequence.

Requirements

To run this code, you need the following:

• Python 3
• OpenAI Python library
• Hugging Face datasets library
• A valid OpenAI API key
• A valid Google API key

Install the required libraries using pip3 and the provided requirements.txt file:

pip3 install -r requirements.txt

Setup

1. Clone this repository:

git clone https://github.com/BenderScript/PromptInjectionBench.git
cd PromptInjectionBench

2. Create a .env file in the project directory and add your OpenAI API and Google Keys:

OPENAI_API_KEY=your_api_key_here
GOOGLE_API_KEY=your_google_api_key_here

## Running the Analysis

To analyze the Jailbreak dataset with ChatGPT-4 and Gemini Pro, execute the following Python script:

```bash
uvicorn prompt_injection_bench.server:prompt_bench_app --reload --port 9002

If Everything goes well, you should see the following page at http://127.0.0.1:9001

This script loads the dataset, iterates through prompts, sends them to ChatGPT-4, and detects potential injection attacks in the generated responses.

Testing

See the demo below where the App checks a prompt with a malicious URL and injection.

Code Structure

• main.py: The main script that loads the dataset, defines functions to interact with ChatGPT-4, and performs the analysis.

Skipping "Benign" Prompts

In the interest of time, the code skips prompts labeled as "benign." This helps focus the analysis on potentially harmful prompts where injection attacks might occur.

Tabulating Attacks vs. Detected Attacks

The script tabulates the total number of generated texts and the number of texts where potential injection attacks are detected. It counts these attacks based on the presence of specific keywords such as "I'm sorry," "apologize," or "developed by OpenAI" in the generated text.

Results

The script will print information about each prompt, including its type, the generated text from ChatGPT-4, and whether an injection attack was detected based on the presence of specific keywords.

License

This code is provided under the Apache License 2.0. Feel free to use and modify it as needed.

---

This analysis is provided as a reference and demonstration of using OpenAI's ChatGPT-4 model for evaluating prompt injection attacks in text datasets.

For more information about OpenAI's GPT-4 model and the Hugging Face Jailbreak dataset, please refer to the official documentation and sources:

• OpenAI GPT-4
• Hugging Face Jailbreak Dataset

These explanations added to the README.md should help users understand why "benign" prompts are skipped and how the code tabulates attacks vs. detected attacks.