pii-masker-ai 🔒🤖
Ultra-fast, zero-dependency PII masking, redaction, and de-identification for AI prompts, LLM agents, and datasets.
Detect and mask supported patterns before sending text to an external model. Review the output: pattern matching cannot guarantee that all sensitive data has been removed.
Start here: your first contribution
Featured beginner issue #11: Write a first redaction walkthrough using synthetic data. Read CONTRIBUTING.md for setup, claiming an issue, and opening a draft PR.
Repository: pii-masker-ai; PyPI distribution: pii-masker-ai; Python import: pii_masker.
The installed CLI is pii-masker.
Try this from a reviewed source checkout, in the repository root, with Python 3.8+. It uses synthetic inputs and needs no API key or network access:
from pii_masker import redact_text
print(redact_text("Contact demo@example.com"))
Expected output:
Contact [EMAIL]
Pattern detection can miss personal data. Review results before sharing them, and keep reversible mappings private because they contain original values.
⚡ The AI Privacy Problem & Solution
sequenceDiagram
participant User
participant PII_Masker as pii-masker-ai
participant LLM as Claude / OpenAI / Gemini
User->>PII_Masker: "Contact John at john@company.com with phone 0912345678"
Note over PII_Masker: Reversible Masking
PII_Masker->>LLM: "Contact John at <EMAIL_1> with phone <PHONE_1>"
LLM->>PII_Masker: "Scheduled an appointment for <EMAIL_1> via <PHONE_1>."
Note over PII_Masker: Automatic Unmasking
PII_Masker->>User: "Scheduled an appointment for john@company.com via 0912345678."
🌟 Key Features
- Zero dependencies: Written in 100% pure Python standard library. No third-party runtime dependencies; this does not eliminate security risks.
- 4 De-identification Modes:
reversible: Sequential numbered tokens (<EMAIL_1>,<PHONE_1>) for LLM chat round-trips.redact: Fixed redaction tags ([EMAIL],[API_KEY]) for audit logs and security reporting.hash: Keyed HMAC-SHA256 pseudonyms (<EMAIL_26d609702036a71b>) for persistent agent memory; consistent across sessions when you set a salt.synthetic: Realistic dummy data from reserved and test ranges (user1@example.com,+1-201-555-0100) for realistic model reasoning.
- Allowlist & Safe Exemption: Exempt exact values, such as a public support email or a known gateway IP (case-insensitive exact match on the detected value).
- LangChain & LLM Chat Wrapper: Wrap any LLM or ChatModel in 1 line of code with automatic prompt masking and response unmasking.
- Batch Dataset Sanitizer: High-speed CLI to sanitize
.jsonl,.csv, and.txtdatasets with column/field targeting. - Comprehensive Pattern Suite (v0.3.0):
- Financial: Credit cards (Visa, MasterCard, Amex, Discover, JCB) with Luhn algorithm verification, IBAN bank codes.
- Network: IPv4, IPv6 addresses, MAC addresses.
- Secrets & API Keys: JWT tokens, PEM Private Keys (
RSA,EC), OpenAI, Anthropic, GitHub, AWS, Slack, Stripe, Google Cloud. - Global & Regional: Email, US SSN, International phone (E.164), Vietnam Citizen ID (CCCD/CMND), Tax ID, Passports.
📦 Installation
pip install pii-masker-ai
🚀 Quickstart (Python API)
1. Reversible Masking for LLMs
from pii_masker import mask_text, unmask_text
# 1. Mask prompt before calling Claude / OpenAI
prompt = "Please send invoice #123 to customer alice@example.com, phone 0912345678."
result = mask_text(prompt, reversible=True)
print(result.masked_text)
# Output: "Please send invoice #123 to customer <EMAIL_1>, phone <PHONE_1>."
# 2. Call your LLM with masked_text...
llm_response = "I have drafted the invoice for <EMAIL_1> and notified <PHONE_1>."
# 3. Restore original customer PII in the response
final_output = unmask_text(llm_response, result.mapping)
print(final_output)
# Output: "I have drafted the invoice for alice@example.com and notified 0912345678."
A repeated value always gets the same token. To mask several texts that share one context
(turns of a conversation, records of a dataset), pass the mapping you have so far: numbering
continues and known values keep their token, so mapping.update(result.mapping) never
overwrites an entry.
from pii_masker import mask_text
mapping = {}
for turn in ["Mail alice@example.com", "Now mail bob@example.com"]:
result = mask_text(turn, mapping=mapping)
mapping.update(result.mapping)
# mapping == {"<EMAIL_1>": "alice@example.com", "<EMAIL_2>": "bob@example.com"}
PIIMasker() loads the Vietnam rules by default (countries=["VN"]); pass countries=[]
to load no country-specific rules.
2. LangChain & Claude Chat Wrapper
from pii_masker.integrations import PIIChatWrapper
from langchain_anthropic import ChatAnthropic
base_model = ChatAnthropic(model="claude-3-5-sonnet-20241022")
# Shield your LLM: automatically sanitizes prompts & unmasks responses
shielded_model = PIIChatWrapper(base_model)
response = shielded_model.invoke("Send email to alice@company.com with bill $500")
print(response.content)
The wrapper masks strings, message objects (text content or content blocks), (role, text)
tuples, dict inputs (values, recursively), lists of these, and prompt values; any other input
raises TypeError rather than being sent unmasked. Each invoke() uses its own mapping, so
one wrapper can serve several users. last_mapping only shows the most recent call and is not
thread-safe.
3. Pseudonymization (Hash Mode) & Synthetic Data
from pii_masker import PIIMasker, MaskMode
masker = PIIMasker(salt="company-secret-salt", allowlist=["support@mycompany.com"])
# Hash mode for persistent agent memory
res_hash = masker.mask("User dev@corp.io reported an issue", mode=MaskMode.HASH)
print(res_hash.masked_text)
# Output (with salt "company-secret-salt"): "User <EMAIL_26d609702036a71b> reported an issue"
# Synthetic replacement
res_synth = masker.synthetic_mask("Customer phone is 0912345678")
print(res_synth.masked_text)
# Output: "Customer phone is +1-201-555-0100"
Hash mode is pseudonymisation, not anonymisation. Tokens are HMAC-SHA256 digests keyed by
your salt, so keep the salt secret: anyone who holds it can guess low-entropy values such as
phone numbers or IDs by hashing candidates. The salt comes from salt=..., else the
PII_MASKER_SALT environment variable, else a random per-instance salt (a warning is shown,
and tokens then differ between processes).
Synthetic values come from reserved or test ranges (example.com emails, 555-01xx phone
numbers, documentation IP ranges, card numbers that deliberately fail the Luhn check) and are
unique for each entity, so unmasking restores every value.
💻 CLI Usage
Masking text
# Reversible masking
pii-masker mask "My email is alice@company.com"
# Redact mode
pii-masker mask "Contact support@example.com" --mode redact
# Output JSON with mapping
pii-masker mask "User 0912345678" --json --save-mapping map.json
# Restore text
pii-masker unmask "Hello <PHONE_1>" -m map.json
# Hash mode with a stable salt (or set PII_MASKER_SALT)
pii-masker mask "User 0912345678" --mode hash --salt "$MY_SECRET_SALT"
Mapping files contain the original values: they are written readable by the owner only and should never be committed.
Batch Dataset Sanitization (JSONL & CSV)
# Sanitize fine-tuning or training datasets
pii-masker batch -i raw_dataset.jsonl -o clean_dataset.jsonl --fields prompt,response --mode redact
# Sanitize CSV customer data
pii-masker batch -i users.csv -o safe_users.csv --fields email,phone --mode synthetic
One mapping covers the whole run, so tokens never collide between records. JSONL values are
masked at any depth (numbers too, when they look like PII); lines that are not valid JSON are
left out of the output, reported on stderr, and make the command exit with code 1. An unknown
CSV column in --fields is an error. The output file must differ from the input file.
Scanning files for PII leaks (CI/CD friendly)
# Returns exit code 1 if any PII leak is found
pii-masker scan ./data/ --strict
scan checks every non-binary file (including dotfiles such as .env) and prints the
category, file:line, and only a short preview of each finding, never the full value.
🌍 Supported PII Categories
| Category | Description | Examples |
|---|---|---|
EMAIL |
Standard RFC email addresses | user@example.com |
PHONE |
International & Vietnamese numbers | +84912345678, 0901234567 |
CREDIT_CARD |
Major credit cards with Luhn verification | 4532-xxxx-xxxx-9012 |
IBAN |
International Bank Account Numbers | DE89370400440532013000 |
IP_ADDRESS |
IPv4 addresses (loopback is not reported) | 198.51.100.1 |
IPV6_ADDRESS |
IPv6 addresses | 2001:db8::1, fe80::1ff:fe23:4567:890a |
MAC_ADDRESS |
Network hardware MAC addresses | 00:1A:2B:3C:4D:5E |
API_KEY |
OpenAI, Anthropic, GitHub, AWS, Slack, Stripe, Google keys, Bearer tokens | sk-..., sk-proj-..., sk-ant-..., ghp_..., AKIA... |
JWT_TOKEN |
JSON Web Tokens | eyJhbGci... |
PRIVATE_KEY |
RSA / EC PEM private keys | -----BEGIN PRIVATE KEY----- |
CREDENTIALS |
user:password@host in URLs of any scheme |
postgresql://user:pass@db.example.com |
SSN |
US Social Security Numbers | 123-45-6789 |
GOV_ID |
National IDs (VN CCCD 12-digit / CMND 9-digit) | 001201012345 |
TAX_ID |
Business tax identification numbers | 0123456789-001 |
PASSPORT |
Passport identification numbers | B1234567 |
🤝 Community: Add Your Country's PII Rules!
We are actively expanding localized PII regex rules for more countries! Country-specific rules need format references, synthetic test cases, and validation:
- 🇫🇷 France (NIR / Carte Vitale)
- 🇯🇵 Japan (My Number)
- 🇩🇪 Germany (Steuer-ID)
- 🇬🇧 UK (National Insurance Number)
See CONTRIBUTING.md to get started.
📄 License
MIT License © 2026 lui01212
Metadata
Release files for pii-masker-ai 0.4.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| pii_masker_ai-0.4.1.tar.gz | 40.4 kB | With provenance | Details for pii_masker_ai-0.4.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| pii_masker_ai-0.4.1-py3-none-any.whl | Python 3 | none | any | Details for pii_masker_ai-0.4.1-py3-none-any.whl |
Total release size: 69.7 kB
Release files / pii_masker_ai-0.4.1.tar.gz
| Download URL | pii_masker_ai-0.4.1.tar.gz |
|---|---|
| Size | 40.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
774d927a1eb32eacdc0449707039fbfbb7c3f412268a5793843511557e50366e
|
|
BLAKE2b-256 checksum How to use checksums |
b1ad13652f032d54032cc1dfb0ad17ec8c857ceb228a94b0c082e8217298e0b7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 16, 2026.
Transparency logRelease files / pii_masker_ai-0.4.1-py3-none-any.whl
| Download URL | pii_masker_ai-0.4.1-py3-none-any.whl |
|---|---|
| Size | 29.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
285aa5d2f97f3c23f6732dd9ec95961885ee88b99ddce21ee8000e682af618dc
|
|
BLAKE2b-256 checksum How to use checksums |
73b0760c47e02ecb4640244fe88853ac4b21c18e67ce49ea09be3217a8a9c6e8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 16, 2026.
Transparency logSecurity for pii-masker-ai 0.4.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for pii_masker_ai-0.4.1.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 16, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.