pii-warden 1.0.0

Privacy-first, hybrid PII detection and redaction engine using local rules and Hugging Face Transformers.

🛡️ PII Warden (Python Package)

Privacy-first, hybrid PII (Personally Identifiable Information) detection and redaction engine for Python applications.

It combines local regex & checksum patterns with Hugging Face Transformers to offer context-aware token classification locally on your machine.

---

Features

• Hybrid Two-Tier Engine:
  • Tier 1 (Instant & Offline): Identifies structured data (Credit Cards, Emails, Phone Numbers, IP Addresses, IBANs) using fast regexes and mathematical checksum validation.
  • Tier 2 (Context-Aware ML): Uses a fine-tuned token classification model (DistilBERT) from Hugging Face to detect names, organizations, and locations.
• Deduplication & Collision Resolution: Automatically merges overlapping matches and resolves boundary clashes between local rules and machine learning predictions.
• Privacy First: Zero API servers or external API hops are required for inference. The model runs locally on your device.

---

Installation

Install the package via pip:

pip install pii-warden

Make sure you have PyTorch installed for your system. If not, pip will attempt to fetch it as a dependency.

---

Quick Start

1. Tier 1: Rules & Checksums Only (Offline, Zero-Dependency)

No model is loaded; it runs instantly using local rules.

from pii_warden import PIIWarden

# Initialize the warden
warden = PIIWarden()

# Analyze text
result = warden.analyze("My credit card is 4111-1111-1111-1111 and email is [e-mail]")

print(result["redacted_text"])
# Output: "My credit card is [ID_1] and email is [EMAIL_1]"

print(result["entities"])
# Output: Detailed list of matched entities (type, start, end, score, source)

2. Tier 2: Hybrid Mode (Local Rules + Transformer Model)

Loads a PyTorch Transformer model from the Hugging Face hub for high-accuracy NER.

from pii_warden import PIIWarden

warden = PIIWarden()

# Load the local model (downloads on first call)
warden.load_model()

# Analyze using hybrid rules + model classification
result = warden.analyze("My name is Samuel Olubukun and my phone number is [phone]")

print(result["redacted_text"])
# Output: "My name is [ID_2] [ID_1] and my phone number is [PHONE_1]"

---

Model Details & Provenance

The ML component of the engine is optimized in two forms:

• Base PyTorch Model: The core classifier is powered by samuelolubukun/pii-ner-finetuned-distilbert. This model is a fine-tune of DistilBERT trained on top of AI4Privacy's high-quality multilingual PII dataset, optimized for Token Classification (NER).
• Edge ONNX Model: If you need to run in web environments or browsers, check out our companion NPM library pii-warden which runs the ONNX optimized version samuelolubukun/pii-ner-edge-optimized.

---

License

MIT