deidgate

Prove your PHI de-identification actually works — before the cloud LLM sees the data.

You told the security review "we de-identify PHI before it hits the model." deidgate is the test that proves it — on every deploy.

It runs a labeled, 100% synthetic HIPAA Safe Harbor corpus through your de-identification function and fails the build if any of the 18 identifiers survive. It runs entirely in your CI and never touches real PHI — so running it does not make you a Business Associate, and there's no BAA and nothing to procure.

pip install deidgate

# tests/test_deid.py
from deidgate import check_all
from myapp.deid import scrub          # <-- your de-identification, whatever it is

def test_deid_strips_all_phi():
    result = check_all(scrub)
    assert result.passed, result.summary()

or gate CI directly:

deidgate check --deid myapp.deid:scrub --md evidence.md   # exit 1 if anything leaks

Why this exists

Every guide says "de-identify first, then call the LLM." Almost nobody independently verifies the de-id works — they trust the vendor's accuracy number. But NLP de-id tops out around 90–99% recall and falls apart on real clinical text: dates written in prose, an MRN embedded mid-word, a reformatted SSN, an age over 89, a hyphenated surname, a nickname. One leak across the BAA boundary is a reportable event.

deidgate is a QA oracle, not another de-id model. Because the corpus is synthetic and labeled, it never has to detect PHI in unknown text (the hard problem everyone loses at). It only checks that each known planted identifier is gone — so the verdict is precise, deterministic, and auditable. It catches a leak three ways:

• exact — the value survives verbatim
• digits — a numeric identifier is reformatted but the digit run survives (123-45-6789 → 123456789)
• split — a multi-token name/address survives with its distinctive tokens intact

What it covers

All 18 Safe Harbor identifiers (deidgate corpus lists them), including the adversarial cases naive pipelines miss:

Try it against the deliberately-imperfect example de-id and watch it fail:

deidgate check --deid examples.naive_deid:naive_scrub
# FAIL — leaks across Safe Harbor categories: date, geo, mrn, ssn, ...

Audit evidence

--md evidence.md / --json evidence.json emits a per-element Safe Harbor pass/fail table you can drop into a validation record:

# deidgate Safe Harbor evidence — ✅ PASS
| # | Safe Harbor identifier            | Tested | Result |
| 1 | Names                             |   3    | ✅ pass |
| 3 | Dates (except year); ages > 89    |   4    | ✅ pass |
| 8 | Medical record numbers            |   3    | ✅ pass |
...

pytest integration

# pyproject.toml
[tool.pytest.ini_options]
deidgate_deid = "myapp.deid:scrub"

from deidgate import check_sample
def test_deid_leaves_no_phi(deidgate_deid, deid_sample):     # one test per corpus sample
    leaks = [l for l in check_sample(deidgate_deid, deid_sample) if not l.warning]
    assert not leaks, "; ".join(f"{l.cat.value}:{l.value!r}" for l in leaks)

Roadmap / commercial packs

The bundled corpus is a free starter set. Maintained adversarial edge-case packs (oncology notes, pediatric dates, i18n names, new LLM-jailbreak-style leaks), drift/canary monitoring, and one-click audit-evidence export are the paid tier — same --pack file.jsonl interface. Interested? → (landing page / waitlist link)

FAQ

• Does it touch real PHI? No. The corpus is entirely synthetic — invented names, fake numbers. That's the point.
• Is this legal or compliance advice? No. It's a testing tool; it gives you evidence, not an attestation.
• What counts as "my de-id"? Any str -> str callable — a regex, a spaCy/Presidio pipeline, a call to a de-id API, whatever you run before the model.

MIT licensed.