redsafe 0.1.0

Local AI-safe redaction engine for security data

AI Safe Redaction Engine

A local-first, modular redaction system for security data before AI processing.

Single-Word CLI Name

This project is packaged for pipx as:

• Command: redsafe
• Package name: redsafe

Other good single-word alternatives if you want to rename later:

• safescrub
• cloaknet
• privashield

Supported Inputs

• Burp Suite XML exports
• Raw HTTP requests/responses
• Network logs
• Basic PCAP parsing
• Screenshots (OCR + masking)

Features

• Analysis-based sensitive data detection (not regex-only)
• Named Entity Recognition (spaCy)
• Secret detection with heuristics and entropy scoring
• Context-aware detection from header/parameter names
• Consistent placeholder mapping (<EMAIL_1>, <JWT_TOKEN_1>, etc.)
• Local-only processing, no external API calls

Project Structure

• parsers/: input ingestion modules
• detection/: entity, secret, entropy, and context detectors
• redaction/: placeholder and redaction logic
• core/: data models and orchestration pipeline
• utils/: file/encoding helpers
• tests/: sample files + pytest coverage

Install With pipx (Recommended)

cd /home/gss/Desktop/Codex/ai-redaction-engine
pipx install .

Run:

redsafe --input tests/sample_burp.xml --type burp
redsafe --input tests/sample_http.txt --type http
redsafe --input tests/sample_log.txt --type log
redsafe --input tests/sample_image.png --type image

To reinstall after local code changes:

pipx reinstall redsafe

Install With venv (Alternative)

cd /home/gss/Desktop/Codex/ai-redaction-engine
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
python -m spacy download en_core_web_sm

Run examples:

python main.py --input tests/sample_burp.xml --type burp
python main.py --input tests/sample_http.txt --type http
python main.py --input tests/sample_log.txt --type log
python main.py --input tests/sample_image.png --type image

Outputs are written to sanitized_output/.

Redaction Tuning (False Positive Control)

Secret entropy detection can be tuned via environment variables:

export REDACTION_ENTROPY_THRESHOLD=4.2
export REDACTION_MIN_SECRET_LEN=24
export REDACTION_MIN_BASE64_LEN=28
export REDACTION_IGNORE_VALUES="application/x-www-form-urlencoded,text/plain"

These values are consumed by SecretDetectionConfig in detection/secret_detection.py.

Run Tests

pytest -q

Example: Burp XML Sanitization

Input Burp request header:

Authorization: Bearer eyJhbGciOiJIUzI1Ni.eyJzdWIiOiIxMjM0NTYifQ.signature123456789
Cookie: sessionid=abcDEF1234567890

Sanitized output:

Authorization: Bearer <JWT_TOKEN_1>
Cookie: sessionid=<SESSION_COOKIE_1>

Example: HTTP Sanitization

Input:

POST /login HTTP/1.1
Host: app.local
Content-Type: application/x-www-form-urlencoded

email=[e-mail]&password=SuperSecret123

Sanitized output:

POST /login HTTP/1.1
Host: app.local
Content-Type: application/x-www-form-urlencoded

email=<EMAIL_1>&password=<PASSWORD_1>

Example: Image Sanitization

• OCR extracts text from screenshot.
• Detection engine flags sensitive values.
• Matching OCR boxes are masked in output image.

Notes

• Designed for integration into a future AI pentesting engine.
• All processing is local.
• If en_core_web_sm is unavailable, regex/heuristic detection still works.
• Image redaction needs local opencv-python, pytesseract, and system tesseract binary installed.