confusable

Unicode spoofing and confusable detection for Python, following Unicode Technical Standard #39. Pure Python, no runtime dependencies.

Status: alpha. Look-alike detection (skeleton, match), mixed-script detection, restriction levels, the identifier checks, the mixed-number and combining-mark checks, invisible-character detection, grapheme clusters and the compatibility module work and pass their tests. The alphas are on PyPI: pip install confusable==0.1.0a2 (pip skips pre-releases unless the version is pinned or --pre is given).

Not to be confused with confusables (1.2.0, from 2021), which matches words that can be confused for each other, or with confusable_homoglyphs (3.3.1), whose API this package provides as confusable.compat.homoglyphs. Every release of confusable is published from this repository's own workflow and carries a provenance attestation on PyPI.

Usage

import confusable

confusable.skeleton("p\u0430ypal")  # 'paypal' (with CYRILLIC SMALL LETTER A)
confusable.match("paypal", "p\u0430ypal")  # Match.MIXED_SCRIPT
confusable.match("rn", "m")  # Match.SINGLE_SCRIPT
confusable.match("paypal", "paypa1x")  # None: not confusable

# The skeleton follows display order, so a right-to-left override is seen
# the way a reader sees it:
confusable.skeleton("invoice\u202egpj.exe")  # 'invoiceexe.jpg'

Restriction levels (UTS #39 section 5.2) rank how safely a string mixes scripts, from ASCII_ONLY to UNRESTRICTED, using the General Security Profile:

confusable.restriction_level("caf\u00e9")  # RestrictionLevel.SINGLE_SCRIPT
confusable.restriction_level("abc\u3042")  # HIGHLY_RESTRICTIVE: Latin with Japanese
confusable.restriction_level(
    "p\u0430ypal"
)  # MINIMALLY_RESTRICTIVE: Latin with Cyrillic
confusable.is_mixed_script("p\u0430ypal")  # True

# Levels are ordered, most restrictive first:
confusable.restriction_level(name) <= confusable.RestrictionLevel.HIGHLY_RESTRICTIVE

The identifier checks follow UTS #39's General Security Profile, under canonical equivalence, so the answer is the same however the text is encoded:

confusable.is_allowed("paypal")  # True
confusable.is_allowed("pay pal")  # False: a space is Restricted
confusable.is_allowed("\u0114")  # True: U+0114 is Restricted, E + U+0306 is not
confusable.identifier_types("\u200d")  # {IdentifierType.DEFAULT_IGNORABLE}
confusable.isxidstart("_"), confusable.isxidcontinue("_")  # (False, True)

# Persian, Malayalam and Sinhala need ZERO WIDTH (NON-)JOINER in some words. The
# profile excludes both; joining_controls=True allows them only where UTS #39
# section 3.1.1.1 says they change how the word looks:
confusable.is_allowed("\u0dc1\u0dca\u200d\u0dbb\u0dd3", joining_controls=True)  # True
confusable.is_allowed("pay\u200dpal", joining_controls=True)  # False

Three more checks from UTS #39 sections 5.3 and 5.4 catch digits from different number systems and combining marks that pile up or hide:

confusable.has_mixed_numbers("1\u0661")  # True: ASCII and Arabic-Indic digits
confusable.has_unlikely_marks("e\u0301\u0301")  # True: the same accent twice
confusable.has_hidden_overlay("i\u0307")  # True: a dot above, lost in i's own dot

Invisible and control characters, for text that is not held to the identifier profile, such as display names and file names:

confusable.find_invisible("pay\u200bpal")  # [(3, Invisible.DEFAULT_IGNORABLE)]: ZWSP
confusable.find_invisible("invoice\u202egpj.exe")  # [(7, Invisible.BIDI_CONTROL)]
confusable.find_invisible("hi\U000e0068\U000e0069")  # two Invisible.TAG: hidden "hi"
confusable.has_invisible(
    "\U0001f468\u200d\U0001f469\u200d\U0001f467"
)  # False: an emoji

Grapheme clusters, the characters a reader sees, and truncation that never cuts one apart. A cluster can be any size, so pair a grapheme limit with a size limit:

[
    str(g) for g in confusable.iter_graphemes("e\u0301\U0001f1fa\U0001f1f8")
]  # ['e' + accent, a flag]
confusable.truncate(name, 32)  # at most 32 graphemes
confusable.truncate(bio, 300, max_utf8_bytes=3000)  # and at most 3000 bytes

skeleton(text, direction) takes the paragraph direction text is shown in: "ltr" (the default, UTS #39's skeleton), "rtl", "auto", or None for UTS #39's internalSkeleton, which ignores display order. Skeletons are for comparison, not display, and change with the Unicode version (confusable.unicode_version), so recompute any you store when it changes.

Moving from confusable_homoglyphs

confusable.compat.homoglyphs has the API of confusable_homoglyphs 3.3.1, so existing code switches by changing one import:

from confusable.compat.homoglyphs import categories, confusables

confusables.is_dangerous("p\u0430ypal")  # True

It returns what that package returns, with current Unicode data. It also finds Arabic and Hebrew look-alikes, which that package's data misses: its generator copies the invisible LEFT-TO-RIGHT MARKs that confusables.txt puts around right-to-left characters in its comments. For new code, prefer match and restriction_level, which follow UTS #39.

What it is for

confusable decides whether user-supplied text is safe to accept as an identity: a username, display name, project name, slug or filename. Planned for v1:

• Look-alike detection with the UTS #39 skeleton algorithm, so that pаypal (with a Cyrillic а) and paypal compare as confusable.
• Restriction levels and mixed-script checks from UTS #39 section 5, built on the Unicode Scripts and Script_Extensions properties.
• Identifier status and type from the UTS #39 General Security Profile.
• Mixed numbers and suspicious combining marks from UTS #39 sections 5.3 and 5.4: digits from different decimal systems, repeated or stacked marks, and a dot above hidden by the letter under it.
• Invisible and control characters: default-ignorable code points, zero-width characters, bidirectional controls, and Tag-block "ASCII smuggling", while still accepting valid emoji tag sequences such as subdivision flags.
• UAX #31 identifier checks equivalent to Python 3.15's unicodedata.isxidstart() and unicodedata.isxidcontinue(), on Python 3.10 and later.
• Grapheme clusters: iteration and truncation that never split a user-perceived character, equivalent to Python 3.15's unicodedata.iter_graphemes().
• A compatibility module, confusable.compat.homoglyphs, with the same API as confusable_homoglyphs 3.3.1, so existing code can switch by changing one import, and that package's tests run against it.

The Unicode data is pinned to a single Unicode version (18.0.0) and generated into the package, so installing needs no network access and no compiler.

Correctness

A look-alike that slips through is a security bug, so the tests come first. Every operation is checked against two independent implementations of UTS #39, ICU and the Rust unicode-security crate, over every Unicode code point and a corpus of known attacks and generated strings. Both oracles run at the pinned Unicode version, and every place where they differ is explained by a specific rule. See tools/oracles/README.md.

Development

uv sync
uv run pre-commit install
uv run pytest
uv run pre-commit run --all-files

CHARTER.md says what the project is for and the principles every change is held to. AGENTS.md has the conventions, for people and coding agents alike.

Security

To report a vulnerability, see SECURITY.md.

License

MIT. See LICENSE. The Unicode data the package and its test fixtures are generated from is under the Unicode License v3; see LICENSES/Unicode-3.0.txt.