Skip to main content

VIPI — Visual Prompt Injection Scanner for Images

Animated VIPI logo reveal

Scan images for prompt injections before an AI agent reads them.
A Python CLI and GitHub Action. Runs fully local or with hosted providers.

Python 3.12 PyPI version OS Support: macOS | Linux | Windows

Images can carry instructions that people overlook but AI models read. VIPI finds the text in an image, including text that standard OCR misses, and checks it for prompt injections. It scans single images, folders and committed image changes.

  • CI/CD and AI code review: scan pull-request images before AI triage tools or coding assistants read them.
  • Image ingestion: scan uploaded or collected images before they enter an AI pipeline.
  • Agents with image input: check screenshots before an agent decides or acts on them.

VIPI scans an image before it reaches an AI agent. Clean images continue to the agent; for images with findings, your workflow reports, blocks or allows them.

Quick start

Requires Python 3.12 and uv.

Local: images never leave your machine.

uv tool install --python 3.12 'vipi-scanner[ocr,detector-local]'
vipi setup --ocr-mode balanced
vipi scan image.png --ocr-mode balanced --output report.json

Cloud: hosted OCR and decisions. Set GOOGLE_CLOUD_API_KEY and OPENAI_API_KEY in the environment or an ignored .env (example).

uv tool install --python 3.12 'vipi-scanner[ocr]'
vipi setup --modus cloud
vipi scan image.png --modus cloud --output report.json

Run setup with the same options you scan with. A bare vipi setup also downloads the large quality model. If vipi is not on PATH, run uv tool update-shell and restart the terminal.

Models at a glance

Every scan runs up to three steps; step 2 runs only when the OCR result is uncertain. The mode decides who provides the models.

VIPI reads the text of an image with OCR, has a visual model review uncertain OCR and re-reads the image if text is missing, then checks the final text for prompt injections. The scan ends clean (exit 0), with an injection (exit 1) or incomplete (exit 2).

Models and memory per mode
Step Local quality (default) Local balanced Local speed Cloud
1 · Read text PaddlePaddle PP-OCRv6 Medium
or Small · Tiny
PaddlePaddle PP-OCRv6 Small
or Medium · Tiny
PaddlePaddle PP-OCRv6 Tiny
or Medium · Small
Google Cloud Vision
or OpenAI · Mistral
2 · Verify OCR visually Cloudflare Clef-Flash 9B
or Liquid AI d1 3B
Liquid AI d1 3B
or Cloudflare Clef-Flash 9B
Liquid AI d1 3B
or Cloudflare Clef-Flash 9B
OpenAI Decisions API
or TypeSafe Jev
3 · Detect injections Cloudflare Clef-Flash 9B
or Liquid AI d1 3B · Meta Prompt Guard 2
Liquid AI d1 3B
or Cloudflare Clef-Flash 9B · Meta Prompt Guard 2
Liquid AI d1 3B
or Cloudflare Clef-Flash 9B · Meta Prompt Guard 2
OpenAI Decisions API
or TypeSafe Jev
Memory, Apple Silicon or GPU ¹ 48 GB unified memory (measured)
or 24 GB VRAM
~16 GB unified memory
or 16 GB VRAM
16 GB unified memory (measured)
or 16 GB VRAM
~8 GB RAM
no GPU needed
Memory, CPU only ¹ ~48 GB RAM ~24 GB RAM ~24 GB RAM ~8 GB RAM

¹ Estimates with the defaults; measured were speed on an Apple M2 with 16 GB and quality on an Apple M4 Pro with 48 GB. Decision models load unquantized: 16-bit on Apple Silicon and CUDA, 32-bit on CPU. Clef-Flash 9B needs ~19 GB in 16-bit and ~38 GB in 32-bit. Disk space, model IDs and pinned revisions are in Models and hardware.

Latest results

Development split of the VIPI evaluation dataset: 227 images with a prompt injection, 223 benign, default threshold 80.

  • Detection rate: injection images reported as an injection. Higher is better.
  • False positive rate: benign images wrongly reported. Lower is better.
  • Words recognized: words in the images VIPI recognized, tolerating a small typo. Higher is better.

Bar charts of detection rate, false positive rate and words recognized: cloud configurations detect 74 to 85 percent of injections with 0.4 to 4.5 percent false positives and recognize 85 to 88 percent of words; local configurations detect 15 to 19 percent with 0 to 2.2 percent false positives and recognize 70 to 77 percent of words. Exact values and models are in the expandable tables below.

Exact values and models
Cloud OCR Detector Detection rate False positive rate Words recognized
Mistral OCR 4.1 TypeSafe Jev 1.14 81.5% 0.4% 84.6%
Mistral OCR 4.1 OpenAI GPT-6 Luna (Decisions API) 85.0% 4.5% 84.6%
OpenAI GPT-6 Luna OpenAI GPT-6 Luna (Decisions API) 83.3% 4.5% 84.8%
OpenAI GPT-6 Luna TypeSafe Jev 1.14 78.9% 1.3% 86.3%
Google Cloud Vision OpenAI GPT-6 Luna (Decisions API, default) 78.4% 3.6% 88.1%
Google Cloud Vision TypeSafe Jev 1.14 74.0% 0.9% 88.1%
Local OCR Detector Detection rate False positive rate Words recognized
PP-OCRv6 Medium (quality) Clef-Flash 9B (default) 16.7% 0% 77.0%
PP-OCRv6 Medium (quality) Prompt Guard 2 86M 19.4% 1.8% 77.0%
PP-OCRv6 Small (balanced) d1 3B 15.4% 0.4% 72.3%
PP-OCRv6 Small (balanced) Prompt Guard 2 86M 16.7% 1.8% 72.3%
PP-OCRv6 Tiny (speed) d1 3B 15.0% 0.4% 70.0%
PP-OCRv6 Tiny (speed) Prompt Guard 2 86M 18.5% 2.2% 70.0%

Usage

vipi scan label.png screenshot.jpg
vipi scan ./images --verbose --output report.json
vipi scan ./images --exclude 'generated/' --exclude '*.webp'
vipi scan . --diff origin/main --output report.json
vipi scan image.png --modus cloud --detector openai --threshold 90
Format Extensions Notes
PNG .png Static images only; animated PNGs are rejected
JPEG .jpg, .jpeg EXIF orientation is applied
WebP .webp Static images only; animated WebPs are rejected

Extensions are matched case-insensitively. Local inputs are limited to 40 megapixels; hosted inputs follow provider limits. Other file types in scanned folders or diffs are skipped and listed in the report.

A score at or above the threshold (default 80/100) is a finding. Scores are not calibrated probabilities, and a score below the threshold does not prove an image is safe. Cloud scans can make additional billable requests.

Exit code Result
0 Scan complete, no findings
1 Injection found, every image completed
2 Error or incomplete scan; wins over findings

Every option, credential and report field is in the CLI reference.

GitHub Actions

Scan the images a pull request adds or changes. This example uses the local balanced profile with Meta Prompt Guard as the injection detector:

name: VIPI image scan
on:
  pull_request:
    branches: [main]
permissions:
  contents: read
jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          fetch-depth: 0
      - uses: EnXan/VIPI@v0.7.2
        with:
          modus: local
          ocr-mode: balanced
          detector: prompt-guard
        env:
          HF_TOKEN: ${{ secrets.HF_TOKEN }}
      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
        if: always()
        with:
          name: vipi-report
          path: vipi-report.json

The runner needs enough memory for the chosen models, and Prompt Guard requires accepted model terms on Hugging Face. Fork PRs normally get no secrets. For hosted models, set modus: cloud and the provider keys; see Action inputs.

Documentation

License

VIPI's code, documentation and assets use the Apache License 2.0. Models, dependencies, datasets and provider marks keep their own terms.

Metadata

Release files for vipi-scanner 0.7.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for vipi-scanner 0.7.2
File Size Uploaded Provenance Actions
vipi_scanner-0.7.2.tar.gz 184.8 kB Without provenance Details for vipi_scanner-0.7.2.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for vipi-scanner 0.7.2
File Interpreter ABI Platform Actions
vipi_scanner-0.7.2-py3-none-any.whl Python 3 none any Details for vipi_scanner-0.7.2-py3-none-any.whl

Total release size: 291.9 kB

Release files / vipi_scanner-0.7.2.tar.gz

Download URL vipi_scanner-0.7.2.tar.gz
Size 184.8 kB
Tags Source
SHA-256 checksum
How to use checksums
ccc754bcf2f7303bfd2e821e1494cb0f95d290890a77b06b5a0f1b7b42d59c1f
BLAKE2b-256 checksum
How to use checksums
350050d9b55ca6145d9e604254024d74aeafbf54681510c01ce5b39cb6af20c5
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.13.5 {"installer":{"name":"uv","version":"0.13.5","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release files / vipi_scanner-0.7.2-py3-none-any.whl

Download URL vipi_scanner-0.7.2-py3-none-any.whl
Size 107.2 kB
Tags Python 3
SHA-256 checksum
How to use checksums
5d89b1fdf1ed87ff744d6236985e5dc1ce50dc8f7717e41952e543a9942f14be
BLAKE2b-256 checksum
How to use checksums
22c1dcaaa68db3865a3f2a5636334f6e452512d36b5fb28b7c0d6c888f543887
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.13.5 {"installer":{"name":"uv","version":"0.13.5","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Security for vipi-scanner 0.7.2

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page