pixmask

Blazing-fast image sanitization for multimodal LLM security. Pure C++ core with SIMD acceleration, Python bindings via nanobind. Zero runtime dependencies.

pip install pixmask

flowchart LR
    subgraph Input
        A[Untrusted\nImage]
    end

    subgraph pixmask ["pixmask Pipeline"]
        B[Validate &\nDecode]
        C[Bit-Depth\nReduction]
        D[Median\nFilter]
        E[JPEG\nRoundtrip]
        B --> C --> D --> E
    end

    subgraph Output
        F[Sanitized\nImage]
    end

    A --> B
    E --> F
    F --> G[Any VLM:\nGPT-4V / Gemini /\nLLaVA / etc.]

Why pixmask?

Every image sent to a multimodal LLM is an attack surface. Adversarial perturbations, steganographic payloads, prompt injection via pixel manipulation, malformed files exploiting parser bugs -- pixmask neutralizes these threats in <15ms with a single function call.

Compared to alternatives

Quick Start

import pixmask
import numpy as np

# One-liner: sanitize any image
safe_image = pixmask.sanitize(image_array)

# From raw bytes (e.g., API upload)
safe_image = pixmask.sanitize(raw_bytes)

# From file path
safe_image = pixmask.sanitize("uploaded_photo.jpg")

# Presets
safe_image = pixmask.sanitize(image, preset="fast")       # ~3ms, bit-depth + JPEG only
safe_image = pixmask.sanitize(image, preset="balanced")    # ~15ms, full pipeline (default)
safe_image = pixmask.sanitize(image, preset="paranoid")    # ~25ms, maximum defense

# Custom parameters
safe_image = pixmask.sanitize(image, bit_depth=4, jpeg_quality=(60, 80))

# Get bytes for API calls
safe_bytes = pixmask.sanitize(raw_bytes, output_format="jpeg")

Integration with VLM APIs

import pixmask

# Sanitize before sending to any VLM
with open("user_upload.jpg", "rb") as f:
    raw = f.read()

safe = pixmask.sanitize(raw, output_format="jpeg")

# Now pass `safe` to your VLM API of choice

How It Works

pixmask applies a multi-stage defense pipeline:

Stage 0: Input Validation

• Magic byte verification (PNG, JPEG, WebP only -- GIF/TIFF/SVG rejected)
• Dimension limits (max 8192x8192)
• File size limits (max 50MB)
• Decompression ratio check (prevents zip/PNG bombs)

Stage 1: Safe Decode

• stb_image with compile-time format restriction (JPEG + PNG only)
• GIF, BMP, TGA, PSD, HDR, PIC, PNM all disabled at build time
• Pixels copied to SIMD-aligned buffer immediately, parser memory freed

Stage 2: Bit-Depth Reduction

• Reduces 8-bit channels to 5-bit (configurable 1-8)
• Implemented with Google Highway SIMD (SSE2/AVX2/NEON)
• Collapses adversarial perturbations that hide in low-order bits
• Destroys LSB steganography as a side effect

Stage 3: Median Filter (3x3)

• 19-step Bose-Nelson sorting network
• SIMD-accelerated via Google Highway
• Removes impulse noise and isolated adversarial pixels
• Edge-preserving for natural image content

Stage 4: JPEG Roundtrip

• Encode to JPEG with randomized quality factor (70-85)
• Decode back to pixel buffer
• Quality randomized per-image using OS entropy (getrandom/getentropy)
• Destroys DCT-domain steganography and high-frequency perturbations
• Randomization prevents adaptive attacks that train through a fixed QF

Architecture

pixmask/
  src/cpp/
    include/pixmask/       # Public C++ headers
      types.h               # ImageView, SanitizeOptions, SanitizeResult
      arena.h               # Zero-allocation bump-pointer allocator
      validate.h            # Stage 0: input validation
      decode.h              # Stage 1: stb_image wrapper
      bitdepth.h            # Stage 2: SIMD bit-depth reduction
      median.h              # Stage 3: SIMD median filter
      jpeg_roundtrip.h      # Stage 4: randomized JPEG roundtrip
      pipeline.h            # Pipeline orchestrator
    src/                    # Implementations + Highway SIMD dispatch
    bindings/module.cpp     # nanobind Python bindings
    third_party/            # Vendored: stb_image, stb_image_write, doctest
  python/pixmask/          # Python package
  src/tests/               # C++ (doctest) + Python (pytest) tests

Design Principles

• Zero runtime dependencies -- numpy is the only optional peer dep
• Pure C++17 core -- no OpenCV, no scipy, no Pillow required
• SIMD everywhere -- Google Highway for portable SSE2/AVX2/NEON
• Arena allocator -- zero heap allocations in the hot path
• Pre-built wheels -- pip install just works, no compiler needed

Building from Source

# Development build
pip install nanobind scikit-build-core[pyproject]
pip install --no-build-isolation -ve .

# C++ only (no Python)
cmake -S . -B build -DBUILD_TESTING=ON
cmake --build build -j$(nproc)
ctest --test-dir build

Testing

# C++ tests
cmake --build build -j$(nproc)
ctest --test-dir build --output-on-failure

# Python tests (after pip install)
pytest src/tests/python/ -v

Roadmap

v0.2

• Bilateral filter (edge-preserving, Pareto-superior to median)
• Gaussian blur (3-pass box blur approximation)
• Haar wavelet denoising (strongest standalone defense)
• Pixel deflection (stochastic, non-differentiable)
• Safe resize with INTER_AREA + random jitter
• Upgrade decoders: libspng + libjpeg-turbo (replacing stb)
• Steganography detection signal (chi-square test)

v0.3+

• OCR-based typographic attack detection
• Total variation denoising (Chambolle-Pock)
• Content-aware adaptive sanitization
• BPDA/EOT adaptive attack evaluation suite

Limitations

pixmask is a preprocessing defense layer, not a complete security solution:

• Typographic attacks (FigStep) embed readable text in images. Pixel-level preprocessing cannot stop this -- OCR-based detection is needed (planned for v0.3).
• Semantic content attacks where the image itself is harmful content require content moderation, not sanitization.
• Fully adaptive white-box adversaries who know the exact pipeline can theoretically bypass any preprocessing defense (Athalye et al., ICML 2018). pixmask is effective against the realistic non-adaptive threat model.

References

• Xu, Evans, Qi -- "Feature Squeezing: Detecting Adversarial Examples", NDSS 2018
• Guo et al. -- "Countering Adversarial Images via Input Transformations", ICLR 2018
• Das et al. -- "SHIELD: Fast, Practical Defense and Vaccination", KDD 2018
• Prakash et al. -- "Deflecting Adversarial Attacks with Pixel Deflection", CVPR 2018
• Qi et al. -- "Visual Adversarial Examples Jailbreak Aligned LLMs", AAAI 2024
• Gong et al. -- "FigStep: Jailbreaking VLMs via Typographic Prompts", AAAI 2025
• Athalye, Carlini, Wagner -- "Obfuscated Gradients Give a False Sense of Security", ICML 2018
• Quiring et al. -- "Adversarial Preprocessing: Image-Scaling Attacks", USENIX Security 2020

License

MIT