rahasya

Thirty-four kinds of personal identifier, each a pattern with a checksum, read in context. 1.000 precision at recall 1.000 on a held-out corpus; 0.996 on the corpus they were tuned against. No model, no network, no gigabyte on disk.

Install

uv add rahasya

The one call

r = pii_report('Invoice for ada@example.com. Card 4111 1111 1111 1111. '
                'KEY=sk-abcdefghijklmnopqrstuvwxyz123456')
r.has_pii, r.identifying

has_pii counts only IDENTIFYING kinds. An IP address is reported and does not tip the gate.

redact('Card 4111 1111 1111 1111 charged to ada@example.com')

Digits are read in context

A number that fails its checksum is not the thing the checksum protects, and a number introduced by a word that makes it a reference is not an identifier at all.

[pii_report(t).has_pii for t in
 ('Order 4111 1111 1111 1112 shipped',   # fails Luhn
  'Conforms to EN 60601-1',              # a standard, not a ZIP
  'Card 4111 1111 1111 1111')]           # this one is a card

DESIGNATOR holds the words that do it: ISO, RFC, invoice, page, commit, order and thirty more. It is matched against the 48 characters to the left of a candidate.

Nineteen regional identifiers, each with its own checksum

Aadhaar, PAN and GSTIN; Australian TFN, ABN and Medicare; Singapore NRIC; Thai national ID; Dutch BSN; French NIR; Spanish DNI and NIE; Italian codice fiscale; Polish PESEL; Swedish personnummer; Norwegian fødselsnummer; German Steuer-IdNr; UK NINO; IMEI.

import random
a = gen_aadhaar(random.Random(0))            # a valid one, for a fixture
a, aadhaar_ok(a), pesel_ok('44051401359'), nino_ok('AB123456C')

Names are opt-in

ner=True turns on an honorific-anchored pass. Dr Charles Babbage matches; a bare Ada Lovelace does not. scanned_ner says whether anything looked, which keeps “none found” apart from “not looked for”.

(pii_report('Dr Charles Babbage signed it.').has_pii,
 pii_report('Dr Charles Babbage signed it.', ner=True).identifying,
 redact('Dr Charles Babbage signed it.', ner=True))

What is here

luhn, aadhaar_ok, pesel_ok and the rest are exported too, because a checksum is useful on its own.

Why no model

Two learned detectors were measured against these patterns and neither is shipped. Both lose on precision and recall, and are 200 times slower. A DeBERTa-v3 ONNX classifier earns its gigabyte only on names no honorific introduces (2/8 to 5/8 of them). A 350M tflite encoder finds no names at all.

Development

The notebooks in nbs/ are the source; the modules are generated.

pip install -e .
nbdev_prepare