presidio-sg-recognizers

Singapore-specific custom recognizers for Presidio. Detects identifiers that Presidio's defaults miss or overshoot in a Singapore context: National Registration Identity Card (NRIC) and Foreign Identification Number (FIN) with full checksum validation, Singapore mobile and landline phone numbers, six-digit postal codes, and configurable internal codes (cost-centre, project, clinical-service, course code, etc.).

The Personal Data Protection Act 2012 (PDPA) is Singapore's data protection law. PDPA-grade detection requires correct, locally-aware recognizers; this package fills the gap that Presidio's out-of-the-box English recognizers leave open for Singapore data.

What's in the box

NRIC and FIN go through full checksum verification, so format-only false positives (random S1234567X-shaped strings that happen to appear in the text) are filtered out. Phone, postal code, and internal code rely on regex format plus Presidio's context boosting; the consuming pipeline is expected to set an appropriate score threshold.

Install

pip install presidio-sg-recognizers

Or with uv:

uv add presidio-sg-recognizers

Requires Python 3.11 or newer, and presidio-analyzer>=2.2,<3 (installed automatically as a dependency).

Use

The fastest path is all_recognizers(), which returns one of each recognizer in a list. Register them against a Presidio AnalyzerEngine:

from presidio_analyzer import AnalyzerEngine
from presidio_sg_recognizers import all_recognizers

analyzer = AnalyzerEngine()
for recognizer in all_recognizers():
    analyzer.registry.add_recognizer(recognizer)

results = analyzer.analyze(
    text="Please verify S0000001I before processing.",
    entities=["SG_NRIC", "SG_FIN", "SG_PHONE", "SG_POSTAL", "INTERNAL_CODE"],
    language="en",
)
for r in results:
    print(r.entity_type, r.start, r.end, r.score)

If you only want a subset, import the per-module factories directly:

from presidio_analyzer import AnalyzerEngine
from presidio_sg_recognizers import sg_nric, sg_fin

analyzer = AnalyzerEngine()
analyzer.registry.add_recognizer(sg_nric.build_recognizer())
analyzer.registry.add_recognizer(sg_fin.build_recognizer())

The internal-code recognizer accepts custom letter and digit counts for organisations whose codes don't fit the 3-letter + 4-digit default:

from presidio_sg_recognizers import internal_code

# e.g. department codes shaped as DEPT12 (4 letters + 2 digits)
recognizer = internal_code.build_recognizer(prefix_letters=4, digit_count=2)

Design notes

Lazy Presidio import

Each recognizer module's presidio_analyzer import is gated behind TYPE_CHECKING at module load time and only imported inside build_recognizer(). That means you can import presidio_sg_recognizers in a lightweight environment (e.g. CI without presidio installed) and only pay the import cost when you actually build a recognizer. The format regexes (NRIC_FORMAT_RE, FIN_FORMAT_RE, etc.) are usable without Presidio at all.

Checksum module is pure-Python, dependency-free

_checksum.py has zero third-party dependencies and is import-safe in any environment. It implements the weighted-sum + prefix-offset + lookup-table algorithm shared by NRIC and FIN. The Presidio PatternRecognizer wrappers in sg_nric.py and sg_fin.py call into it from validate_result() to drop format-correct but checksum-invalid candidates.

F, G, and M share a single FIN lookup table

Earlier community write-ups sometimes claim three distinct M-series tables; cross-checking against fifteen samples confirmed valid by an independent online tool (see test_checksum.py) disproves that. F, G, and M differ only in the prefix offset added before the modulo step; the trailing-letter lookup table is the same.

Context-aware postal codes

A bare 6-digit number is too ambiguous to flag on its own (phone last-six, reference numbers, year-month concatenations, the like). The postal recognizer ships with a low base score (0.3) and a context list of street suffixes, "Singapore", "Blk", and similar. Presidio's analyzer engine boosts the score when those context words appear within the proximity window, so address-shaped surroundings raise the postal candidate into the detection range while isolated 6-digit runs stay below it.

Testing

uv sync --group dev
uv run pytest -ra
uv run ruff check .
uv run ruff format --check .

The test suite has three parts:

1. Checksum tests (tests/test_checksum.py): hand-verified fixed fixtures, samliew-cross-validated samples, and a Hypothesis property-based pass that exercises the full (prefix, digits) input space.
2. Format-regex tests (tests/test_format_regex.py): bare-regex behaviour for every recognizer, runnable without presidio-analyzer installed.
3. Presidio integration tests (tests/test_recognizers_integration.py): construct real PatternRecognizer instances and call analyze() directly. Skipped cleanly when presidio-analyzer isn't installed.

Versioning

Follows Semantic Versioning. Public API for v0.x is:

• Module names: sg_nric, sg_fin, sg_phone, sg_postal, internal_code.
• Entity type strings: SG_NRIC, SG_FIN, SG_PHONE, SG_POSTAL, INTERNAL_CODE.
• Per-module build_recognizer() factories and *_FORMAT_RE re-exports.
• The top-level all_recognizers() convenience.

Breaking changes to any of the above will bump the major version once the package reaches 1.0.

License

Apache License 2.0. See LICENSE.

The patent grant in Apache 2.0 is the reason this package ships under Apache rather than MIT (the underlying pdpa-scrub project that motivated it uses MIT). Library-style packages benefit from explicit patent terms.

Attribution

See ATTRIBUTIONS.md for dependency credits and the provenance stance on the NRIC/FIN checksum algorithm.

Contributing

Issues and pull requests are welcome. Before opening a PR:

• Tests pass (uv run pytest -ra).
• Code is formatted (uv run ruff format .) and lint-clean (uv run ruff check .).
• No real Singapore identifiers, names, or organisations in fixtures or examples. Use placeholders that are visibly fabricated: a repeated digit, an ascending or descending run, or zero-padded (S1111111D, S1234567D, S0000001I, "Jane Doe"). PRs that introduce random-looking values will be asked to swap them out.

Why the fixture rule is about shape, not origin

A checksum-valid identifier is checksum-valid whoever holds it. Generating one at random does not make it unused, and nothing in the number records where it came from, so a reader cannot tell a fabricated identifier from one issued to a real person by looking at it. The rule therefore asks for placeholders that are obviously fabricated, not merely synthetic ones.

This repository did not always meet its own rule. An earlier set of fixtures was generated rather than harvested, and was still random-looking, which is the exact case the rule exists to prevent: the provenance was fine and the result was indistinguishable from a real number anyway. They were replaced with repeated-digit forms in #2. That history is written down rather than dropped, because a rule reads as arbitrary until you know which mistake produced it.

For larger changes (new recognizer types, breaking API changes, new ways to configure existing recognizers), open an issue first to talk through the design.

Context

This package was extracted from pdpa-scrub, a two-stage anonymiser for PDPA and intellectual property material. The Singapore recognizers are useful on their own (anywhere a Presidio pipeline runs against Singapore-context text), so they live here as a separate Apache 2.0 library.