maskrelay

Self-hosted, reversible, entity-consistent PII tokenization for LLM traffic.

Text sent to an LLM has PII replaced with stable typed placeholders — every mention of the same person ("Charlie", "Charlie Douglas", "Mr. Douglas") becomes one [PERSON_1], a different person becomes [PERSON_2] — so the model can still reason about who did what. LLM output is rehydrated back to the original values before anyone sees it. Raw PII never leaves infrastructure you control.

client ──► tokenize: detect (ensemble) → span-merge → alias linker → encrypted vault ──► LLM
client ◄── detokenize: tolerant ladder / streaming hold-back buffer ◄──────────────────  LLM

Three traffic shapes

1. Real-time chat/agents — OpenAI-compatible proxy (POST /v1/chat/completions) with streaming response detokenization: a placeholder split across SSE chunks (…[PER / SON_1]…) is still restored mid-stream, byte-identical to the batch path. Tool-call arguments are tokenized on the way out and rehydrated on the way back.
2. Batch documents — two-pass tokenization (mode="batch") with retroactive alias merging via union-find before any token is issued.
3. Structured records — column-policy YAML + JSON path rules; the same value maps to the same token across files under one dataset scope, so joins survive tokenization (orders.csv ⋈ customers.csv on tokenized keys == the raw join, row-for-row).

Quick start

uv sync                      # core + dev deps (includes the small spaCy English model)

# in-process round trip
uv run pii demo --text """Bank Rep: Good morning, thank you for calling [Bank]. How can I assist you today?

Customer: Hi, yes, I've lost my debit card and I need to report it.

Bank Rep: I'm sorry to hear that. I can definitely help you with that. Before we proceed, I'll need to verify your identity for security purposes. Can I start by confirming your full name as it appears on your account?

Customer: It's Sarah Jane Mitchell.

Bank Rep: Thank you, Sarah. And just to confirm, your date of birth?

Customer: 14th of March, 1987.

Bank Rep: Perfect. And can you confirm the address we have on file for you?

Customer: It's 42 Riverside Court, Manchester, M2 5DB.

Bank Rep: Excellent, thank you. That all matches our records. Now, when did you last have your card, so we know when to block it from?

Customer: Sometime yesterday afternoon, I think. I used it at the supermarket around 3 PM, and then I realized it was gone later that evening.

Bank Rep: Right, so we'll block it as of yesterday at 3 PM to be safe. I'm cancelling that card immediately, and you won't be able to use it. The good news is we'll have a replacement sent to you within 5 to 7 working days. In the meantime, you can still access your account through mobile banking or online, and you can use contactless payments on your phone if you have that set up.

Customer: Great, thanks. And I should check for any unauthorized transactions, shouldn't I?

Bank Rep: Absolutely. Have a look through your recent transactions and report anything suspicious to us right away. But given you only lost it yesterday evening, it's unlikely anyone's used it yet. Is there anything else I can help you with today?

Customer: No, that's all. Thank you.

Bank Rep: You're welcome, Sarah. Your replacement card is on its way. Have a great day!"""

# durable round trip across processes (SQLite vault, encrypted at rest)
export PII_VAULT_KEK="$(python -c 'import base64,os; print(base64.b64encode(os.urandom(32)).decode())')"
uv run pii tokenize   --backend sqlite --db vault.db --session s1 --text "…"
uv run pii detokenize --backend sqlite --db vault.db --session s1 --text "…[PERSON_1]'s reply…"

# service + OpenAI-compatible proxy
uv sync --extra service
PII_SVC_AUTH_TOKEN=devtoken PII_SVC_UPSTREAM_BASE_URL=https://api.openai.com \
PII_SVC_UPSTREAM_API_KEY=sk-… \
  uv run uvicorn --factory maskrelay.service.app:create_app --port 8000
# point your OpenAI client at http://localhost:8000/v1 — PII never reaches the upstream

# evaluation harness + CI gates
uv sync --extra eval
uv run python -m maskrelay.eval --preset fast --n-docs 100 --markdown report.md

Detection tiers

# standard tier (downloads the quantized ONNX model on first use)
uv sync --extra transformer
uv run pii demo --preset standard --transformer-repo openai/privacy-filter \
  --file transcript.txt            # add --ignore DATE to keep dates readable
                                   # (note: that also un-protects birth dates)

On the bank-call transcript in the docs, the standard tier captures the full address ("42 Riverside Court, Manchester, M2 5DB") as one [ADDRESS_1], links "Sarah Jane Mitchell"/"Sarah" to one [PERSON_1], and suppresses spaCy's sentence-start ORG noise — the fast tier's documented gaps.

All native labels map through one canonical taxonomy; unmapped labels are dropped with a metric, never passed through.

Design pillars

• Split-biased alias linking: mentions merge only on deterministic, high-precision evidence (name parts, initials, honorifics, particles, suffixes, gender hints). A wrong merge would restore the wrong person's name — silent corruption; a wrong split only costs utility. Every restored string literally occurred in the input.
• Vault-backed reversibility: session/dataset-scoped vault (memory / SQLite / Redis) with per-scope envelope encryption (AES-256-GCM + HKDF subkeys); lookup keys are per-scope HMACs; deleting a session crypto-shreds it (incl. a WAL checkpoint so nothing lingers in the SQLite -wal); session meta is encrypted too. Backups contain ciphertext only; a pre-deletion backup still holds the wrapped DEK, so pair erasure with backup-retention limits.
• Digits are never fuzzy-matched: detokenization tolerates case-mangling, spacing, possessives, markdown wrapping, bracket-style drift and type typos ([PRESON_1]), but [PERSON_1] can never claim [PERSON_2] or [PERSON_11].
• Unknown tokens stay verbatim: an LLM-invented [PERSON_9] is surfaced as a metric, never guessed at.
• Irreversible classes: credit cards and passwords are redacted to [REDACTED:TYPE] and can never be restored, by policy.
• No PII in logs, mechanically: a structlog allowlist processor replaces any non-allowlisted field with <unlogged>; a CI canary test plants PII and asserts it never appears in captured logs.
• Request-owned state: all mapping state lives on an explicit RequestContext; pipelines are stateless singletons. The service refuses multi-worker deployments with the in-memory vault (per-worker divergence — use Redis).

Measured behavior (fast preset, own harness — see caveat)

Synthetic corpus: overall strict P 0.945 / R 0.961; EMAIL/PHONE/SSN/CREDIT_CARD/IP/URL recall 1.000; PERSON F1 0.963; ADDRESS 0.000 (spaCy has no address entity — this is the documented fast-tier gap the standard/max tiers exist to close). Adversarial corpus: lenient P 0.800 / R 0.831. Round-trip fidelity and streaming equivalence: 100%. Alias wrong-merge rate: 0.000 (wrong-split 0.089 — the deliberate side of the bias). Tokenize p50 ≈ 5 ms/doc, detokenize p50 ≈ 0.04 ms.

Caveat: numbers are measured on this repo's own synthetic + adversarial corpora against its own taxonomy. They are not comparable to published benchmarks. Published cross-domain PII benchmarks put all off-the-shelf detectors dramatically lower than their in-domain numbers; run the harness on your own data before relying on it.

Compliance tooling

docs/ciso-brief.md (security briefing), docs/dpia-annex.md (DPIA input), docs/operations.md (runbooks). Implemented controls: KMS-capable key custody (WrappingKeyProvider — the KEK never enters the process; AwsKmsKeyProvider reference under [aws]), rotate_kek() re-wrap, server-enforced retention ceilings + tenant-wide crypto-shred erasure, hash-chained tamper-evident audit log (pii audit-verify), minimization policy templates (--policy-template strict-minimization: identifiers destroyed, dates coarsened to years), model revision pinning (--transformer-revision), and a scheduled eval drift ratchet (python -m maskrelay.eval --ratchet ...).

Compliance note

Pseudonymization/GDPR/HIPAA statements in the docs are engineering inferences, not legal advice; have counsel review before making compliance claims. Wider vault scopes (cross-session consistency) increase linkability and are opt-in.

License

MIT — see LICENSE.