probegpt 0.1.0

Automated red-team discovery for AI models

outofthebox

Automated red-team discovery for AI models. Generates probe prompts, sends them to a target model, judges responses against your objective, and iteratively refines the best-scoring probes.

Install

pipx install probegpt

Usage

probegpt

Full-screen TUI. Set up your models in Config, then go to Run Discovery, enter your objective, and hit Run.

Models

Supports two providers per role (target / generator / judge):

• Azure OpenAI — uses az login credentials automatically
• OpenRouter — set OPENROUTER_API_KEY in a .env file or your environment

Config is saved to ~/.outofthebox/config.json.

How it works

1. Loads 9 seed probes covering known red-team techniques
2. Iteration 1 — generator writes N probes targeting your objective
3. Iteration 2+ — top-scoring probes from the previous round seed mutation + fresh objective variants
4. Each probe is sent to the target model; the judge scores the response against your objective (0.0 → 1.0)
5. Results optionally exported to JSON

Requirements

• Python 3.11+
• For Azure: az login with access to your deployment
• For OpenRouter: OPENROUTER_API_KEY env var