redactor

Detect and reversibly mask PII / secrets in text before you send it to an LLM — then un-mask the model's reply.

Part of the ragkit suite. Install with pip install ragkit-redactor, then import redactor.

redactor scans text for things you probably don't want leaving your machine — emails, phone numbers, SSNs, credit cards, IP addresses, URLs, API keys — swaps them for stable placeholders like [EMAIL_1], and gives you a mapping to put the originals back afterward. Send the redacted text to a third-party AI API, get a response that still references the placeholders, and restore it locally.

• Pure Python standard library. No dependencies.
• Python 3.8+.
• Reversible round-trip by design (restore(redact(t).text) == t).
• Luhn-validated credit-card detection to cut down false positives.
• Pluggable custom detectors with optional validators.

Install

pip install ragkit-redactor

Local development (from redactor/):

pip install -e .

Quick Start

import redactor

text = (
    "Hi, this is Alice. Reach me at alice@example.com or +1 (555) 123-4567. "
    "My card is 4111 1111 1111 1111 if you need to charge the deposit."
)

result = redactor.redact(text)

print(result.text)
# Hi, this is Alice. Reach me at [EMAIL_1] or [PHONE_1].
# My card is [CREDIT_CARD_1] if you need to charge the deposit.

print(result.mapping)
# {'[EMAIL_1]': 'alice@example.com',
#  '[PHONE_1]': '+1 (555) 123-4567',
#  '[CREDIT_CARD_1]': '4111 1111 1111 1111'}

# --- send result.text to your LLM of choice ---
llm_reply = "Sure — I'll confirm the deposit and email [EMAIL_1] the receipt."

# Un-mask the model's output locally:
print(result.restore(llm_reply))
# Sure — I'll confirm the deposit and email alice@example.com the receipt.

The redacted text is safe(r) to send to a third-party API; the sensitive values never leave your process.

API Reference

redactor.redact(text, **kwargs) -> RedactionResult

Convenience wrapper: builds a Redactor(**kwargs) and redacts text in one call.

redactor.luhn_check(number_str) -> bool

Runs the Luhn checksum over the digits in number_str (ignoring spaces/dashes). Used internally to validate credit-card candidates; exposed for your own use.

class Redactor(detectors=None, placeholder_template="[{type}_{n}]", mask_char=None)

• detectors — list of detector names to enable. Defaults to all built-ins. Pass [] to start empty and add your own.
• placeholder_template — format string with {type} and {n} fields. Default "[{type}_{n}]" produces [EMAIL_1], [PHONE_2], etc.
• mask_char — if set (e.g. "*"), matches are replaced by that character repeated to the length of the match. This mode is one-way (see below).

add_detector(name, pattern, validator=None)

Register a custom detector. pattern is a regex string or compiled pattern. validator(str) -> bool optionally filters raw matches (return False to reject). Custom detectors take priority over built-ins for overlap resolution.

r = redactor.Redactor()
r.add_detector("EMP_ID", r"EMP-\d{4}", validator=lambda s: s != "EMP-0000")

detect(text) -> List[Match]

Returns all matches across enabled detectors, with overlaps resolved and sorted by start offset. Overlap resolution is deterministic: earlier start wins; for the same start the longer span wins; remaining ties go to detector priority (registration order, custom detectors first).

redact(text) -> RedactionResult

Detects, then substitutes placeholders (or mask characters). Identical values get the same placeholder (dedupe). Output is built by walking matches left-to-right, so offsets never corrupt.

restore(text, mapping=None) -> str

Replaces placeholders in text with their originals using mapping (required — a bare Redactor keeps no state). Longer placeholders are substituted first so [X_1] can't be clobbered by [X_11].

class Match

Dataclass: type (detector name), value (original text), start, end (offsets in the source), placeholder (assigned during redact, else None).

class RedactionResult

• .text — the redacted string.
• .matches — list of Match with placeholders assigned.
• .mapping — {placeholder: original_value} (empty in mask_char mode).
• .restore(model_output) — un-mask placeholders in an LLM response using this result's mapping. This is the reversible round-trip.

Built-in Detectors

False positives & Luhn validation

Regex PII detection is a balance between catching real data and not flagging every number in your prose. A few pragmatic choices:

• Credit cards are only reported when the digits pass luhn_check, which discards the vast majority of random 13–16 digit strings.
• API keys / secrets are kept deliberately narrow (sk- tokens and long hex blobs) rather than "anything that looks random", to avoid swallowing ordinary words and IDs.
• Overlaps (e.g. an IP embedded inside a URL) collapse to a single match, so you never get doubled or garbled output.

You will still see occasional misses and occasional false hits. Tune with the detectors list and add_detector.

Reversible placeholders vs. mask_char

There are two modes:

• Placeholder mode (default, reversible). Values become [TYPE_n] tokens and a mapping is returned. You can restore() both your text and the LLM's reply. restore(redact(t).text, mapping) == t.
• mask_char mode (one-way). Set mask_char="*" and each match becomes **** of equal length. No mapping is produced and restoration is impossible — use this only when you want to scrub data for display/logging, not round-trip it.

r = redactor.Redactor(mask_char="*")
print(r.redact("SSN 123-45-6789").text)   # SSN ***********

A note on compliance

This library is best-effort. Regex-based detection cannot catch every form of sensitive data, and matching data is not the same as understanding it. redactor is a practical safety net, not a guarantee of GDPR / HIPAA / PCI-DSS compliance or any other regulatory requirement. Review the redacted output before relying on it, and don't treat it as a substitute for proper data-handling controls.

License

MIT.