credgate

Detects personal data before it leaves the machine, with Indian identifiers built in: Aadhaar (with Verhoeff checksum), Aadhaar Virtual ID, PAN, GSTIN, voter ID, passport, driving licence, UPI IDs, IFSC, bank accounts, payment cards, +91 mobile numbers, ABHA, and names and addresses in Latin and Indic scripts.

The detection rules live in a shared, language-neutral spec, so this package and the R package credgate find exactly the same matches. Part of the CredGate project; early development.

Use it with local LLMs (Ollama, vLLM)

Redact before the prompt reaches the model; the model only ever sees placeholders such as [PAN] and [PERSON_NAME].

import credgate, ollama

note = "Customer name: Priya Nair. PAN ABCPN1234K, mobile +91 98765 43210, wants the EMI date moved."
print(credgate.preview(note, min_confidence=0.5))   # what will be masked, and why
safe = credgate.redact(note, min_confidence=0.5)    # "Customer name: [PERSON_NAME]. PAN [PAN], mobile [MOBILE_IN], ..."
reply = ollama.chat(model="gemma3:1b", messages=[{"role": "user", "content": f"Summarise: {safe}"}])

With vLLM (or Ollama's /v1, LM Studio, any OpenAI-compatible server):

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")
client.chat.completions.create(model="<served model>",
                               messages=[{"role": "user", "content": credgate.redact(note, min_confidence=0.5)}])

From the shell: credgate redact notes.txt | ollama run gemma3:1b "Summarise:". More runnable examples (ticket classification, RAG ingestion) are in the examples folder.

Check the preview. Names are found next to an honorific, relation or field label (Shri, S/o, Name:, नाम:); a name in running text without such a cue ("Customer Priya Nair called") is not detected yet. Redaction is irreversible; to get the real values back in the model's answer, tokenise instead (see below). An automatic proxy is on the roadmap.

How it compares

_{About the data: the 700 documents are synthetic - KYC forms, bank messages, call-centre notes, Hinglish chats, Hindi text and hospital summaries generated by benchmark/generate_corpus.py and included in the repository under benchmark/corpus/. No real personal data was used. Types both tools cover, exact span matching, thresholds tuned on a separate dev split for every system; time is for 1,400 documents. Synthetic data generated by us, so read with that in mind.}

Where Presidio does better. credgate is built for Indian identifiers and Indian-language context, and that is what this benchmark measures. Presidio is broader and more mature:

• Voter IDs without context: Presidio scores 1.000 here, credgate 0.677 (credgate reports a bare voter ID format at low confidence, 0.3, because it could equally be a product code).
• Many more entity types: international IDs (US, UK, EU, ...), IBAN, IP and MAC addresses, URLs, crypto wallets, dates, nationality and religion - credgate detects none of these.
• Names in running text: Presidio's NER finds names without an honorific, field label or "ji"; credgate does not yet.
• Anonymisation: Presidio can mask, hash or encrypt with a key, and add custom operators; credgate redacts, or tokenises with an in-memory vault that can be saved encrypted (no hashing or encryption operators).
• Beyond text: image (OCR) and structured-data redaction, a REST service and Docker images.
• Maturity and extensibility: years in production, many languages and NLP back-ends, custom recognisers; credgate is pre-alpha.
• The corpus favours credgate: we generated it; on general English text Presidio may well do better.

Full method, per-type results and how to reproduce: benchmark/README.md. The comparison is independent and not affiliated with Microsoft.

Installation

Python 3.11 or later; depends only on regex and PyYAML.

pip install credgate

The development version: pip install "git+https://github.com/dwarpal/credgate#subdirectory=pkg/python".

With Nix: nix build github:dwarpal/credgate#credgate.

Usage

import credgate

spans = credgate.detect_pii("Name: Priya Nair; PAN: ABCPN1234K; UPI: priya.n@oksbi")
for s in spans:
    print(s)
# Span(doc=0, start=6, end=16, type='person_name', value='XXXXX XXXX', confidence=0.8)
# Span(doc=0, start=23, end=33, type='pan', value='XXXXXX234K', confidence=0.97)
# Span(doc=0, start=40, end=53, type='upi_id', value='XXXXX.X@Xksbi', confidence=0.99)

spans[1].value    # 'ABCPN1234K' - printing masks values, the attribute does not
spans[1].reasons  # "matched PAN pattern; context 'PAN'; holder type: individual"

detect_pii() takes a string or a list of strings and returns Span objects with the document index, 0-based end-exclusive positions, type, data class, value, canonical form, confidence and reasons. span.to_dict() gives a JSON-ready dict with the value masked (reveal=True to include it).

Redact identifiers, or preview what would be redacted and why:

credgate.redact("Name: Priya Nair; PAN: ABCPN1234K")
# 'Name: [PERSON_NAME]; PAN: [PAN]'

print(credgate.preview("Name: Priya Nair; PAN: ABCPN1234K; order no 878245671885"))
# 3 items would be redacted:
#   Name: [PERSON_NAME]; PAN: [PAN]; order no [AADHAAR]
#
#   [PERSON_NAME]  XXXXX XXXX    0.80  matched Person name pattern
#   [PAN]          XXXXXX234K    0.97  matched PAN pattern; context 'PAN'; ...
#   [AADHAAR]      XXXXXXXX1885  0.42  ...; negative context 'order'

Tokenise instead, to get the real values back in a model's answer: the vault keeps the mapping, and restore() puts the values back:

vault = credgate.Vault()
credgate.tokenise([
    "Shri Ramesh Kumar, PAN ABCPE1234F, mobile +91 98765 43210",
    "pan abcpe1234f called again from 9876543210",
], vault)
# ['Shri [PERSON_NAME_1], PAN [PAN_1], mobile [MOBILE_IN_1]',
#  'pan [PAN_1] called again from [MOBILE_IN_1]']

credgate.restore("[PERSON_NAME_1] (PAN [pan_1]) will be called on MOBILE_IN_1.", vault)
# 'Ramesh Kumar (PAN ABCPE1234F) will be called on +91 98765 43210.'

The same identifier gets the same token however it is written, so the model can still tell customers apart. restore() accepts tokens as models tend to copy them (other case, spaces, missing brackets); strict=True raises UnknownTokenError on a token the vault does not know. The vault prints with values masked.

To restore tokens in a later session - or in R - save the vault to a file encrypted with a passphrase (at least 12 characters; from the argument, the CREDGATE_VAULT_PASSPHRASE environment variable, or a prompt):

vault.save("loans.cgvault")                    # pip install 'credgate[vault]'
vault = credgate.Vault.open("loans.cgvault")   # later: same tokens, numbering continues

The passphrase is never stored: without it the file cannot be opened. Keep *.cgvault files out of version control.

Scan a table (a pandas DataFrame, or a dict of column -> values) to see which columns hold PII; column names count as context:

report = credgate.scan_pii({
    "cust_pan": ["ABCPE1234F", "AAACR5055K", "XYZPQ9876R"],
    "notes": ["Called, Aadhaar 542229371182", "no issues", "mobile 9876543210"],
})
print(report)
# Scanned 2 column(s); PII found in 2: cust_pan, notes
#   cust_pan  pan              100% of 3 (3 whole value)  confidence 0.97  e.g. XXXXXX234F
#   notes     aadhaar           33% of 3 (0 whole value)  confidence 0.97  e.g. XXXXXXXX1182
#   notes     mobile_in         33% of 3 (0 whole value)  confidence 0.93  e.g. XXXXXX3210
pandas.DataFrame(report.to_records())  # as a table

Context makes generic patterns trustworthy; pass a column name or similar:

credgate.detect_pii("ABC1234567")                      # [] - could be anything
credgate.detect_pii("ABC1234567", context="voter_id")  # voter ID, confidence 0.97

Checksums, synthetic test data and your own detectors:

credgate.verhoeff_valid("542229371182")
credgate.synth_indian_pii(5, seed=1)   # synthetic records; never use as anonymised data

credgate.register_detector(credgate.Detector(
    id="employee_id", data_class="internal", label="Employee ID",
    patterns=credgate.bounded("EMP[0-9]{6}"),
))

Large data: pandas, Polars, Arrow, DuckDB, Parquet

redact(), tokenise() and detect_pii() accept a pandas or Polars Series or an Arrow array and return the same type (a Series keeps its index and name; missing values stay missing). For tables, install the extra you need: pip install 'credgate[polars]', 'credgate[arrow]' or 'credgate[duckdb]', or everything (these, plus encrypted vaults) with pip install 'credgate[all]'.

import polars as pl
import credgate.polars  # adds the .credgate namespace

df.with_columns(pl.col("notes").credgate.redact(min_confidence=0.5))
df.filter(pl.col("notes").credgate.has_pii())

import duckdb, credgate.duckdb

con = duckdb.connect()
credgate.duckdb.register(con, min_confidence=0.5)   # adds cg_redact() and cg_has_pii()
con.sql("COPY (SELECT id, cg_redact(notes) AS notes FROM 'calls.parquet') TO 'safe.parquet'")

import credgate.arrow
credgate.arrow.redact_parquet("calls.parquet", "safe.parquet", columns=["notes"])

credgate redact --parquet calls.parquet --column notes -o safe.parquet
credgate scan calls.parquet

See the large-data guide for the approach and runnable examples. Parquet files, DuckDB queries and Polars lazy frames are processed in batches, so the data may be larger than memory. scan_pii() also accepts Polars frames (eager or lazy), Arrow tables and datasets, and DuckDB relations; it reads one column at a time and samples exactly as it does in memory, so the report is the same at any size. Numbered tokens are not offered inside Polars and DuckDB, which process batches in parallel and in no fixed order; tokenise a Series or Arrow array instead.

Command line

credgate detect notes.txt                      # masked report
credgate detect --lines --json customers.csv   # one JSON object per match
git diff --cached | credgate detect --fail     # non-zero exit if PII is found
credgate redact notes.txt > notes.redacted.txt  # replace PII with [PAN] etc.
credgate preview notes.txt                     # what would be redacted, and why
credgate scan customers.csv                    # which columns hold PII
credgate detectors                             # list detectors

Values are masked unless you pass --reveal.

Masked output

Wherever credgate shows a detected value (printed matches, preview(), scan_pii() examples, the command line), it masks it:

Keeping the last 4 characters lets you check which value matched, and stays within the display limits of UIDAI's masked Aadhaar and PCI DSS (cards). If your policy allows no part of a value to be shown, mask everything:

credgate.set_mask("full")   # or the environment variable CREDGATE_MASK=full

Masking changes only what is displayed; detection and redact() ([AADHAAR], [PERSON_NAME]) are unaffected.

Limits

Names and addresses are found only next to a cue: an honorific (Shri, Smt, श्रीमती, or "ji" after the name), a relation (S/o, D/o), a field label (Name:, पता:) or a house/village keyword followed by a PIN code. A deity addressed with "ji" ("Ram ji") is taken as a name. A bare 12-digit number may pass the Aadhaar checksum by chance and is reported with lower confidence.

Citing credgate

If you use credgate in your work, please cite it, with the version you used (credgate.__version__):

Sikka B, Bais A (2026). credgate: Detect Indian Personal Data Before It Reaches Language Models. Python package version 0.5.0. https://github.com/dwarpal/credgate

@Manual{credgate,
  title  = {{credgate}: Detect Indian Personal Data Before It Reaches Language Models},
  author = {Bharat Sikka and Aishwarya Bais},
  year   = {2026},
  note   = {Python package version 0.5.0},
  url    = {https://github.com/dwarpal/credgate},
}

Licence

Apache License 2.0.