injection-bench

A small, dependency-free harness for evaluating prompt-injection and jailbreak detectors the way they will actually be used: trained on the past, tested on the future, with confidence intervals and shortcut baselines. It exists because detectors that look strong on author-written examples often fall apart on real traffic.

What it does that a quick script usually doesn't:

• Time split with leakage control. Train on earlier items, test on later ones. Near-duplicate prompts (same normalised 300-character prefix) are removed, and no test prompt may duplicate a train prompt.
• Operating-point metrics. Recall at fixed false-positive rates (1%, 5%), AUC, and precision at a stated prevalence, each with a 95% bootstrap confidence interval. A 0.5 threshold on an arbitrary score says little.
• Shortcut baselines. length-only and a hand-written keywords detector show how much of a score is explained by dataset artefacts.
• Plug in your own detector. Anything with score(list[str]) -> list[float] (and an optional fit).

Quick start

pip install injection-bench            # add [sklearn] for the tfidf-lr baseline
injectbench download data/                      # public JailbreakHub CSVs (Shen et al., CCS 2024, MIT)
injectbench run --data data/ --detector length --detector keywords --detector hashed-lr --detector tfidf-lr --out report.md

Your own detector:

# mydet.py
class MyDetector:
    name = "mine"
    def fit(self, texts, labels): ...          # optional
    def score(self, texts): return [...]       # higher = more likely an attack

injectbench run --data data/ --detector mydet:MyDetector

Your own data: a JSONL file with text, label (1 = attack) and date fields: --data my.jsonl.

Reference results

JailbreakHub in-the-wild prompts, 1,405 jailbreaks and 13,735 regular prompts (rows without a date are dropped). Train 7,895 prompts (757 jailbreaks) on the earlier part, test 5,503 prompts (398 jailbreaks, 7.2% prevalence) on the later part. Reproduce with the command above plus --detector guardkit:rules --detector guardkit:model (needs pip install guardkit).

Reading it: length alone gives AUC 0.59, so there is a mild shortcut but not a dominant one. Classical bag-of-words detectors plateau at AUC 0.74-0.78 and catch roughly one in ten jailbreaks at a 1% false-positive rate; adding character n-grams raises AUC but not low-FPR recall. Regular expressions are close to chance on this data. Transformer-based detectors are not included because the benchmark environment could not download model weights; adding one is a ten-line adapter (see "Plug in your own detector").

Caveats

• "recall @X% FPR" sets the threshold on the test negatives. It measures ranking quality at a given false-positive budget; it is not a deployable operating point, which you must choose on held-out data.
• The only bundled dataset labels jailbreak intent in user prompts, not document-embedded prompt injection. Its "regular" class includes templates that open with "ignore previous instructions". Use --data with your own labelled data for your threat model.
• Pure-Python metrics are fine for thousands of items, not millions. Bootstrap CIs use 200 resamples by default.
• Reference rows come from one dataset and one split. Do not read them as a ranking of detector families.

Related projects

• guardkit: the detector used for the guardkit:* rows above; its technical report (docs/technical-report.md) explains why author-written test sets overstate performance.
• bharatid: validators for Indian identifiers.

MIT licensed.