privacy-filter

PII redaction for Python using the OpenAI privacy-filter model. Detect and redact personally identifiable information (PII) from text before sending to LLMs or storing in logs.

Features

• Local PII detection — Uses the openai/privacy-filter HuggingFace model (runs locally, no API calls)
• Reversible redaction — Replace PII with placeholders like [EMAIL_1], [PERSON_2], then restore original values later
• Entity types — Email, phone, address, person names, URLs, dates, account numbers, secrets
• Configurable thresholds — Adjust confidence scores and filter by entity type
• Thread-safe — Lazy-loaded singleton pipeline with double-checked locking

Installation

pip install privacy-filter

Requires Python 3.11+.

Quick Start

from privacy_filter import get_classifier, redact_text, unredact_text, PiiStore

# Load model (first run downloads ~50MB, cached after)
classifier = get_classifier()

# Detect PII
text = "My email is alice@example.com and my name is Alice Smith"
entities = classifier(text)

# Redact
store = PiiStore()
redacted = redact_text(text, entities, store)
print(redacted)  # "My email is [EMAIL_1] and my name is [PERSON_1]"

# Unredact (restore original values)
restored = unredact_text(redacted, store)
print(restored)  # "My email is alice@example.com and my name is Alice Smith"

API Reference

get_classifier(cache_dir=None)

Returns a HuggingFace token-classification pipeline. First call downloads the model (~50MB). Thread-safe singleton.

redact_text(text, entities, store, min_score=0.8, entity_types=None)

Replaces detected entities with placeholders. Populates store.forward with placeholder → original mappings.

unredact_text(text, store)

Restores original values from placeholders using regex substitution.

PiiStore

Dataclass holding forward (placeholder→value dict) and counters (per-type counters).

Supported Entity Types

Advanced Usage

Filter by entity type

entities = classifier(text)
redacted = redact_text(text, entities, store, entity_types=["private_email", "private_phone"])

Adjust confidence threshold

redacted = redact_text(text, entities, store, min_score=0.95)

Custom cache directory

classifier = get_classifier(cache_dir="/path/to/cache")

Integrations

• NanoBot — pip install nanobot-privacy-filter-hook
• OpenAI Agents SDK — pip install openai-agents-privacy-filter

License

MIT