pii-protector 2.2.5

Production-grade PII detection with multi-model ensemble

pii-protector

Production-grade PII (Personally Identifiable Information) detection library with a sequential multi-model ensemble. Designed for speed — 90%+ of texts are handled by the regex layer alone in under 3ms, with heavier NER models invoked only when needed.

Architecture

Layer 1 — Regex                  always runs              ~0.3–3ms
    ↓  escalation score >= 3?
Layer 2 — Presidio + spaCy NER   names / orgs / loc       ~5ms
    ↓  confidence low or conflict?
Layer 3 — NER Transformer        high-accuracy names      ~20ms
    +
Layer 4 — PII Model              structured PII           ~15ms

Each layer decides whether the next one is needed. On clean structured text, only Layer 1 runs.

Installation

# Regex layer only — zero dependencies
pip install pii-protector

# + Presidio / spaCy NER (Layer 2)
pip install "pii-protector[presidio]"
python -m spacy download en_core_web_sm

# + Transformer models (Layer 3 & 4, CPU)
pip install "pii-protector[transformers]"

# + Transformer models (GPU / CUDA)
pip install "pii-protector[gpu]"

# Full install (all layers)
pip install "pii-protector[full]"
python -m spacy download en_core_web_sm

Quick Start

from pii_detector import AdvancedPIIDetector

# Regex only (fastest, zero dependencies)
detector = AdvancedPIIDetector(use_presidio=False, use_transformers=False, use_pii_model=False)

results = detector.detect("My SSN is 123-45-6789 and email is [e-mail]")
for entity in results:
    print(entity)
# {'entity_type': 'SSN',   'start': 10, 'end': 21, 'score': 0.95, 'text': '123-45-6789'}
# {'entity_type': 'EMAIL', 'start': 36, 'end': 52, 'score': 0.95, 'text': '[e-mail]'}

Full ensemble (all layers)

detector = AdvancedPIIDetector()

# Detailed analysis with risk score
analysis = detector.get_detailed_analysis("Dr. John Smith's Aadhaar is 1234 5678 9012")
print(analysis['risk_assessment'])
print(analysis['timing'])

Batch detection

texts = ["call me at [phone]", "my PAN is ABCDE1234F", "nothing sensitive here"]
results = detector.detect_batch(texts)

Risk assessment

risk = detector.get_risk_assessment("SSN 123-45-6789, VISA 4111 1111 1111 1111")
# {'risk_level': 'CRITICAL', 'score': 0.95, 'high_value_pii': ['SSN', 'CREDIT_CARD']}

CLI

# Basic scan
pii-detect "My credit card is 4111 1111 1111 1111"

# From file
pii-detect --file document.txt

# JSON output
pii-detect --json "Aadhaar: 1234 5678 9012"

# Detailed analysis
pii-detect --detailed "SSN 123-45-6789, John Smith, [e-mail]"

# Risk score only
pii-detect --risk "GSTIN 22ABCDE1234F1Z5, PAN ABCDE1234F"

# Disable heavy models for speed
pii-detect --no-presidio --no-transformer --no-pii-model "card: 4111111111111111"

Supported PII Types

Category	Types
Core	EMAIL, PHONE, SSN, CREDIT_CARD, IP_ADDRESS, ADDRESS, PASSWORD, USERNAME
India	AADHAAR, PAN, GSTIN, UPI_ID, IFSC, VOTER_ID, MOBILE_IN, EPFO_UAN, DRIVING_LICENSE_IN, PASSPORT_IN, BANK_ACCOUNT_IN, PINCODE
North America	DRIVER_LICENSE, PASSPORT, BANK_ACCOUNT, TAX_ID, SIN, ZIP_CODE, POSTAL_CA
NER	NAME, ORGANIZATION, LOCATION, FACILITY
Finance	CREDIT_CARD, BANK_ACCOUNT, MONEY
Dates & URLs	DATE_OF_BIRTH, DATE, URL
API Secrets	AWS_ACCESS_KEY, OPENAI_API_KEY, GITHUB_TOKEN, GITLAB_TOKEN, STRIPE_KEY, JWT_TOKEN, BEARER_TOKEN, PRIVATE_KEY, DATABASE_URL, SLACK_TOKEN, DISCORD_TOKEN, 20+ more

Configuration

detector = AdvancedPIIDetector(
    use_presidio=True,           # Layer 2: spaCy NER via Presidio
    use_transformers=True,       # Layer 3: NER transformer
    use_pii_model=True,          # Layer 4: PII model
    confidence_threshold=0.5,   # Early-stop threshold
    spacy_model="en_core_web_sm",
    transformer_model="jainsatyam26/pii-ner-onnx",
    pii_model="jainsatyam26/pii-detector-onnx",
)

Environment variable overrides

SPACY_MODEL_NAME=en_core_web_sm
TRANSFORMER_MODEL_NAME=jainsatyam26/pii-ner-onnx
PII_MODEL_NAME=jainsatyam26/pii-detector-onnx
SPACY_DATA_PATH=/path/to/models

Models

Layer	Model	What it detects
2	en_core_web_sm via Presidio	NAME, ORG, LOCATION
3	jainsatyam26/pii-ner-onnx	High-accuracy person names
4	jainsatyam26/pii-detector-onnx	PHONE, EMAIL, SSN, DOB, ADDRESS

ONNX models (HuggingFace) — both ship FP32 and FP16 variants:

• jainsatyam26/pii-ner-onnx — NER model (model.onnx / model_fp16.onnx)
• jainsatyam26/pii-detector-onnx — PII structured model (model.onnx / model_fp16.onnx)

FP16 is loaded automatically on CPU for ~25% faster inference; falls back to FP32 if unavailable.

License

MIT — tensoryug