RednBlue CLI v3.3.1

Zero-Knowledge Adversarial Security Testing for AI Models

RednBlue CLI is a command-line tool for testing the adversarial robustness of machine learning models and the security posture of LLM/RAG/Agent systems. Run security assessments locally — your model never leaves your infrastructure.

███████████               ███████████
▒▒███▒▒▒▒▒███             ▒▒███▒▒▒▒▒███
 ▒███    ▒███  ████████    ▒███    ▒███
 ▒██████████  ▒▒███ ▒▒███  ▒██████████
 ▒███▒▒▒▒▒███  ▒███  ▒███  ▒███▒▒▒▒▒███
 ▒███    ▒███  ▒███  ▒███  ▒███    ▒███
 █████   █████ ████  █████ ███████████
▒▒▒▒▒   ▒▒▒▒▒ ▒▒▒▒  ▒▒▒▒▒ ▒▒▒▒▒▒▒▒▒▒▒

Zero-Knowledge Adversarial Security Testing

---

Features

• Zero-Knowledge Protocol — Model weights and data never leave your infrastructure
• Image Classifiers — Test ResNet, VGG, EfficientNet, and custom architectures
• YOLO Detection — Full support for YOLOv5, YOLOv8, YOLOv10, YOLOv11
• LLM / RAG / Agent Red-Team Suite — Jailbreak resistance, system prompt extraction, indirect prompt injection, token-level obfuscation, RAG pipeline security, agentic tool-hijacking/goal-drift/privilege-escalation, backdoor auditing, and model extraction resistance — against hosted providers (OpenAI/Anthropic/Gemini), a local Python target file, or any HTTP endpoint
• Tier-Based Testing — Freelancer (quick scan) and Enterprise (comprehensive)
• Visual Explanations (XAI) — See exactly which pixels confused the model, including the real perturbed image, not a placeholder
• Blue Team Hardening — rnb defend applies a local defensive layer to a vulnerable model
• Encrypted Submission — AES-256 encrypted results with HMAC-SHA256 signing
• Signed Compliance Reports — Map test results onto EU AI Act, NIST AI RMF, and ISO/IEC 42001 articles, with a tamper-evident trust seal
• Multi-Jurisdiction Compliance — EU AI Act, NIST AI RMF, ISO/IEC 42001, UK DSIT, Canada AIDA, Singapore MAIGF

---

Installation

# Install from PyPI
pip install rednblue

# Verify installation
rnb

Requirements

• Python 3.8+
• PyTorch 2.0+ (for classifier/YOLO testing)
• CUDA (optional, for GPU acceleration)
• For LLM testing against a hosted provider: pip install openai / pip install anthropic / pip install google-generativeai as needed — not bundled by default, since most users only test one provider

---

Quick Start

1. Choose your interface

Desktop UI (recommended for non-CLI users):

rnb ui

Command line: see below.

2. Set your token (always required for rnb llm; required for certified/submitted runs on rnb preview)

# Linux/Mac
export RNB_TOKEN=RB-XXXXXX-YYYYYY

# Windows
set RNB_TOKEN=RB-XXXXXX-YYYYYY

Get a token at https://dashboard.rednblue.io/dashboard/credits.

3. Run a free preview (Classifier / YOLO)

Image Classifier:

rnb preview --model resnet50.pth --input ./test_images --model-type classifier

YOLO Detection Model:

rnb preview --model yolov10n.pt --input ./test_images --model-type yolo

4. Submit for certification (Classifier / YOLO)

rnb preview --model yolov10n.pt --input ./images --model-type yolo --submit

5. Run an LLM / RAG / Agent security assessment

Against a hosted provider:

rnb llm --provider openai --model gpt-4o-mini --api-key sk-xxxxx --submit

Against a local Python file exposing a chat function (auto-discovered, or specify --function):

rnb llm --file my_target.py --submit

Against an HTTP endpoint:

rnb llm --endpoint http://localhost:8000/chat --input-field prompt --output-field response --submit

Restrict to specific attack classes (see Assessment Dimensions below for codes):

rnb llm --file my_target.py --attacks JB,SPE,TM --submit

6. View your report

Go to: https://dashboard.rednblue.io/dashboard/tests to download your PDF report and certificate, for both Classifier/YOLO and LLM/RAG/Agent sessions.

---

Commands

Command	Description
rnb	Show welcome banner and quick start
rnb-welcome	Same as rnb — convenience alias usable outside the main group
rnb ui	Launch the desktop graphical interface
rnb preview --help	Run adversarial attacks (classifier or YOLO)
rnb llm --help	Run the full LLM/RAG/Agent adversarial security assessment
rnb status	Check CLI version and token validity/tier
rnb explain --help	Generate a standalone visual-explanation panel for one image (Robust Integrated Gradients)
rnb defend --help	Apply local defensive hardening to a model (Blue Team)
rnb report --help	Generate a signed compliance report + trust seal from saved classifier/YOLO results
rnb optimize-epsilon	Optimize epsilon values (Enterprise)

---

Supported Model Formats

Type	Extensions
PyTorch	.pt, .pth, .safetensors
ONNX	.onnx
Ultralytics YOLO	.pt (auto-detected)

Architectures: ResNet, VGG, DenseNet, EfficientNet, MobileNet, Inception, SqueezeNet, ShuffleNet, GoogLeNet, AlexNet, and common timm backbones.

TensorFlow (.h5, .pb) is not yet supported. The ingestion layer (rnb/core/ingestion/wrapper.py) is architected to accept a TensorFlow adapter, but tensorflow is intentionally not bundled as a dependency today — adding real support is tracked as a follow-up, not claimed here.

LLM/RAG/Agent targets: hosted providers (OpenAI, Anthropic, Gemini), a local Python file exposing a chat(user_message: str) -> str-style function, or any HTTP endpoint accepting a JSON prompt field and returning a JSON response field.

---

Assessment Dimensions

Classifier Models

Dimension	Description
Noise Resilience	Stability under sensor noise and interference
Spatial Consistency	Robustness to spatial feature shifts
Universal Pattern Defense	Resistance to universal perturbation patterns
Feature Stability	Internal representation integrity
Confidence Calibration	Prediction reliability accuracy
Iterative Stress Tolerance	Defense against sustained pressure
Optimization Attack Defense	Resistance to optimized adversarial inputs
Deep Perturbation Resistance	Resilience against deep layer perturbations

YOLO Detection Models

Dimension	Description
Noise Resilience	Stability under sensor noise
Input Perturbation Defense	Resistance to subtle input modifications
Iterative Stress Tolerance	Defense against multi-step attacks
Detection Consistency	Reliable detection under varying conditions
Targeted Evasion Defense	Resistance to deliberate misclassification
Object Persistence	Maintains detections under perturbations
Multi-Object Stability	Accuracy in crowded scenes
Black-Box Resilience	Defense without model access
Query-Limited Defense	Resistance to low-query probing

LLM / RAG / Agent

Each class runs against the target's actual responses — no attack ever counts as a success unless the response shows positive evidence of engaging with the malicious request (see v3.3.0 Changelog entry below); a target that's unreachable or merely non-committal is never scored as compromised.

Code	Class	Sub-attacks
JB	Jailbreak Resistance	Template Battery, Many-Shot Jailbreaking, Crescendo Multi-Turn, Multilingual Jailbreak, Persona Transfer, Competing Objectives
SPE	System Prompt Protection	Direct Extraction, Differential Probing, Token Fishing
LPI	Indirect Injection Defense	Document Body Injection
TM	Token Manipulation Robustness	Unicode Homoglyph Substitution, Zero-Width Character Injection, Leetspeak Substitution, Context Dilution, Bidirectional Text Override
RAG	RAG Pipeline Security	Knowledge Boundary Probe, Corpus Poisoning (requires inject_fn), Context Window Stuffing, RAG Prompt Injection
AGT	Agentic Safety	Tool Hijacking via Injected Content, Goal Override / Objective Drift, Privilege Escalation Chain
BD	Backdoor Integrity	STRIP Consistency Test, Trigger Enumeration, Input Sensitivity Probe
EXT	Model Extraction Resistance	Extraction Feasibility Assessment, Response Consistency Test, Model Fingerprinting

Use --attacks with a comma-separated list of codes to restrict a run (e.g. --attacks JB,SPE,TM). Omit --attacks to run all classes.

---

Tier Comparison

Feature	Freelancer	Enterprise
Classifier Attacks	5	8
YOLO Attacks	4	9
Epsilon Values	2	4
Total Scenarios	~10-20	~30-70
LLM / RAG / Agent Testing	❌	✅ (all 8 classes, 27 sub-attacks)
Epsilon Optimization	❌	✅
Continuous Integrity Monitoring	❌	🔜 Coming soon

---

Output Example

============================================================
  RednBlue Security Preview — YOLO Detection
============================================================
  Attacks run    : 21
  Successful hits: 0/21 (0%)
  Robustness rate: 100%
  Estimated Grade: GOLD

  ⚠️  This is a preview only
  → Visit: https://rednblue.io/checkout
  → Re-run with: rnb preview --model-type yolo --submit

---

Certification Grades

Grade	Score	Meaning
🥇 GOLD	≥90%	Excellent robustness, deployment ready
🥈 SILVER	≥75%	Good robustness, minor improvements recommended
🥉 BRONZE	≥50%	Moderate robustness, improvements needed

---

Architecture

For the complete, AI-readable source dump of this repository, see rnb.md.

---

Links

• Platform: https://dashboard.rednblue.io
• Documentation: https://docs.rednblue.ai
• Website: https://rednblue.io

---

Authors

• Dr. Mahdi Deramgozin — Chief AI Officer
• Dr. Saeid Samizade — Chief Technology Officer

---

License

Proprietary — RednBlue SAS © 2026

Made in France 🇫🇷

---

Changelog

[3.3.1] — 2026-07-06

Fixed

• TM-BIDI (Bidirectional Text Override) was written but never wired into token_manipulation.py's run_attack(), so it never actually ran despite being listed in the module's own docstring. Now wired in as a fifth Token Manipulation sub-attack.
• The Issue #37 false-positive fix (v3.3.0) is now extended to the remaining attack modules that shared the same vulnerable pattern: rag_attacker.py's Knowledge Boundary Probe and extraction_attacker.py's Model Fingerprinting both used a raw "not refused and long enough" check, which — like the jailbreak.py case fixed in v3.3.0 — misclassified transport errors and generic, on-topic-unrelated deflections as genuine findings. Both now require positive evidence (topical overlap with the probe, not just absence of refusal) via the same shared is_compliant_response() / is_generic_deflection() / has_topical_overlap() helpers.
• token_manipulation.py's shared _run_variant() (used by all five TM sub-attacks) had the same vulnerable pattern and is now fixed identically.
• Restored a working-copy fix to agent_attacker.py that had been accidentally reverted to a syntactically-broken committed version (unescaped newlines inside string literals) during a repository cleanup pass — no logic change, this is a syntax-only restoration.

Not changed in this release (same class of gap, lower priority — see the modules' own logic for why): agent_attacker.py's three sub-attacks and backdoor_attacker.py's three sub-attacks already use more specific keyword-match or baseline-comparison logic and are less prone to this exact failure mode.

---

[3.3.0] — 2026-07-02

Fixed

• Critical: --tier manual override on rnb llm/rnb preview bypassed RNB_TOKEN validation entirely, allowing unlimited unpaid runs. Token is now always required and validated.
• Critical: API/transport errors (e.g. invalid API key) were scored as successful jailbreaks. Added target health pre-check and error-aware refusal detection; runs now abort cleanly on unreachable targets instead of producing fabricated results.
• High: Generic, harmless, non-refusal responses were scored as successful jailbreaks purely for not containing refusal language. Added positive-evidence (topical relevance) classification to jailbreak.py and prompt_injection.py's Token Fishing.
• Removed the stale RNB_BETA_LLM_SUBMIT gate — rnb llm --submit no longer requires a beta flag now that Dashboard v3.1 supports LLM sessions.
• Desktop UI (rnb ui) was reading stale dimensions/avg_reliability fields that don't exist in LLMTestRunner's actual attacks/success output shape, causing LLM test results to always display as fully safe regardless of actual findings.

Changed

• --tier on rnb llm/rnb preview now only warns on mismatch instead of silently overriding the token's actual tier.

---

[3.1.0] — 2026-06

Fixed

• "After Attack" panel no longer mirrors the original image. The real adversarial tensor produced by the attack engine now flows through to the XAI panel for both classifier and YOLO models, instead of the attention map being computed against an unmodified copy of the input.

Added

• Unified model-ingestion layer (rnb/core/ingestion/wrapper.py) — a BaseModelWrapper abstraction with a PyTorch adapter (covering .pt/.pth/.safetensors) and an ONNX adapter, replacing ad-hoc format detection with a single, extensible interface. TensorFlow is an architected-but-not-yet-implemented extension point (see note above).
• rnb defend — Local Blue Team hardening. Reads a vulnerability profile and registers forward hooks (spatial smoothing + gradient clipping) on the target model's convolutional layers to dampen adversarial perturbations at inference time, with zero retraining.
• rnb explain — Standalone XAI playground for a single (model, image) pair. Adds a RobustIntegratedGradientsExplainer, which averages gradients over many noised copies of the input so the attention map itself can't be blinded by an adversarial perturbation the way single-pass saliency methods can.
• rnb report — Generates a signed compliance report from saved rnb preview --save-results output: maps Noise Resilience, Jailbreak Bypass Rate, and Bias Score-style metrics onto EU AI Act and NIST AI RMF articles, then emits a Markdown report plus a signed trust_seal.svg, both protected with the same AES-256-CBC + HMAC-SHA256 scheme already used for result submission.
• --save-results PATH flag on rnb preview, so a preview run's JSON output can be fed into rnb report later without re-running the attacks.
• rnb-welcome entry point now actually works (previously pointed at a function that did not exist in cli.py).

Changed

• rnb/banner.py no longer hard-codes its own version string — it now imports from rnb/_version.py, the single source of truth.
• Repo cleanup: removed BANNER_INTEGRATION.md, patch_banner.py, the stray root-level banner.py, rnb_init.py, rnb/architecture.md (an old AI-context dump that was accidentally being shipped inside the installable package), rnb/.installed, and test.py (a manual integration-test script that contained a hardcoded API token).
• Documentation consolidated: QUICKSTART.md and CHANGELOG.md have been folded into this README. The only other Markdown file in this repository is rnb.md, the AI-readable full-source dump.

Deferred

• Continuous Integrity Monitoring Daemon (rnb daemon) is not included in this release. It is reserved for a future Enterprise-only release.

---

[3.0.0] — 2026-05-08

Major release. Added the rnb/xai/ visual-explanation package (GradCAMExplainer, EigenCAMExplainer, worst-case selection, 3-panel composite rendering), the desktop UI (rnb ui, Eel-based), GPU auto-detection, the --device and --tier flags, and multi-architecture classifier support (ResNet, VGG, DenseNet, EfficientNet, MobileNet, Inception, SqueezeNet, ShuffleNet, GoogLeNet, AlexNet). Fixed tier propagation to classifier attacks, the run_preview() input_path / Fore import bugs, and token-validation credit lookups.

[2.5.1] — 2026-04-14

Added HuggingFace whitelist filtering and timm backbone support. Migrated the build system from setup.py to pyproject.toml.

[2.5.0] — 2026-04 (initial public release)

Welcome banner, rnb status, rnb preview, rnb optimize-epsilon, rnb test-llm. Eight image-classifier attacks (GNI, SHFP, UAP, FSP, CCM, PGD, CW, DEEP). AES-256-CBC + HMAC-SHA256 encrypted result submission to the dashboard.

Full per-commit history is available via git log. This section retains only release-level highlights.