dv_normalizer — written Dhivehi to spoken Dhivehi

Converts written Dhivehi (Thaana script) into the spoken form: numbers, dates, times, money, percentages, abbreviations, URLs and the literary sentence endings of news prose (…އެވެ) become the words a Dhivehi speaker would actually say. Built as the text-normalization frontend for Dhivehi text-to-speech, and usable on its own.

ލިޔެފައިވާ ދިވެހި ބަސް ވާހަކަދައްކާ ދިވެހި ބަހަށް ބަދަލުކުރާ ނިޒާމު

What it's for:

• Dhivehi text normalization — written to spoken Dhivehi conversion of numbers, dates, money, percentages, and more, in the Thaana script
• Dhivehi number to words conversion (cardinals, ordinals, fractions, decimals, years)
• The text-normalization frontend for a Dhivehi TTS (text-to-speech) pipeline, or standalone use in any Maldivian language NLP tool

Installation

pip install dv-normalize

Quick start

from dv_normalize import normalize

normalize("ވަކި ލާރިން ވެސް 232.23 ލާރި ހޯދައެވެ")
# 'ވަކި ލާރިން ވެސް ދުވިސައްތަ ތިރީސް ދޭއް ޕޮއިންޓް ތޭވީސް ލާރި ހޯދައި'

normalize("ޑރ. އިބްރާހިމް 14:30 ގައި އައި")
# 'ޑޮކްޓަރު އިބްރާހިމް ސާދަ ގަޑި ތިރީސް ގައި އައި'

normalize("ކ.އަތޮޅު ވިލިނގިލިން 120 ކިލޯ މީޓަރު")
# 'ކާފު އަތޮޅު ވިލިނގިލިން އެއް ސަތޭކަ ވިހި ކިލޯ މީޓަރު'

normalize("ފޭސް2ގެ")
# 'ފޭސް ދޭއް ގެ'

normalize("އިތުރު މައުލޫމާތު: https://example.com/news/123 ބައްލަވާ")
# 'އިތުރު މައުލޫމާތު: https://example.com/news/123 ބައްލަވާ'  (URL passes through untouched)

For repeated use, hold onto a Normalizer instance:

from dv_normalize import Normalizer, NormalizerConfig

n = Normalizer(NormalizerConfig(keep_punctuation=False))
n("ހެލޯ، ދުނިޔެ")  # → 'ހެލޯ ދުނިޔެ'

# strict=True raises ValueError if any digit survives normalization; on
# well-formed input it's a no-op safety net.
ns = Normalizer(NormalizerConfig(strict=True))
ns("232 ލާރި")  # → 'ދުވިސައްތަ ތިރީސް ދެ ލާރި'

How it works

A four-stage pipeline: tokenize (split text into words, punctuation and whole-passthrough spans like URLs) → classify (a priority-ranked rules engine tags each token with a semiotic class — CARDINAL, DATE, MONEY, …) → verbalize (each class's own module renders the spoken-form words) → post-process (sentence-ending rewriting, spacing, quote/punctuation fixups). The number and date rules were checked against a 2-million-plus- sentence news + majlis corpus; the 214 sentence-ending rules were reviewed by a native speaker one family at a time against real corpus sentences and cross-checked against ~440,000 majlis speech sentences.

What it handles

The classifier is priority-ranked, so more specific patterns (calendar markers, multi-letter compound abbreviations, year ranges, the digit-anchored symbol rules) shadow the generic ones. Tokens that don't match any rule pass through unchanged.

Sentence-ending coverage: 214 reviewed rewrite rules handle the literary noun/verb ending families found across the news + majlis corpora (formal -އެވެ/-ެވެ endings collapsed to their spoken form). The rules were validated against 437,000 majlis speech sentences — majlis transcripts are themselves largely spoken-register, so a rule that also matches the majlis corpus's own usage is strong evidence it is producing a real spoken form rather than an invented one.

Coverage

Measured against a 1.79M-sentence news corpus and a 399k-sentence majlis (parliament) corpus.

• Full-corpus baseline, before the 1.0 ending-rules and symbol-handling work: news 96.87% of 1,794,272 sentences fully normalized, majlis 99.44% of 399,088 sentences.
• Final 6,000-document sample (3,000 news articles + 3,000 majlis turns) for this release, measured two ways:
  • Raw (every sentence in the sample, Dhivehi or not): news 98.18%, majlis 99.93% fully normalized.
  • Dhivehi-content (sentences that contain at least some Thaana text; a sentence whose only non-Dhivehi content is a passed-through URL still counts as handled): news 99.86%, majlis 99.93%.
  • Zero sentences in either sample left a digit unspoken.

The raw-sample residue is overwhelmingly English-language prose and web-scraping boilerplate (menus, bylines, "read more" links) that isn't Dhivehi text to begin with, so it's outside what a Dhivehi normalizer is meant to convert. The smaller Dhivehi-content residue is mostly English words embedded inside otherwise-Dhivehi sentences (left as-is per unknown_latin), street-address-style slashes, and Arabic-script Quranic or hadith citations.

Ending alignment: of literary-register endings found in the news sample, 99.9% are rewritten by some rule at all, and 91.5% are rewritten to a form that also occurs, verbatim, as a genuine spoken-register ending somewhere in the majlis corpus. That second number is a deliberately conservative metric — a rewritten ending that happens not to appear in the majlis sample isn't necessarily wrong, it just wasn't independently confirmed against spoken-register evidence.

Known issues

Measured state of 1.0.0 (see Coverage for how the numbers were taken): 99.86% of Dhivehi-content news sentences and 99.93% of majlis sentences come out fully spoken, no digit is ever left unread, and 91.5% of rewritten sentence endings are attested verbatim in spoken-register majlis speech. In a human review of 28 random news articles read end to end, 20 needed no change and 8 had corrections; every one of those corrections is now encoded as a rule. What remains:

• Definite vs. indefinite nouns. Literary Xއެވެ (definite, "it is X") and Xެކެވެ (indefinite, "an X") are handled separately, but a few legacy overrides still blur them: ދައްކައެވެ reads ދައްކަ while the spaced ދައްކަ އެވެ reads ދައްކައި. Awaiting a native-speaker ruling.
• ގެ after a number is always read as the genitive particle, so a number before ގެ meaning "house" inflects wrongly (ގޭ is not produced for ގެ އެވެ either).
• Ablative ން after a number is not inflected: 5 ން ފެށިގެން reads ފަސް ން rather than the spoken ފަހަކުން.
• Durations in mm:ss after a hour count keep the colon: 3 ގަޑި 30:04 މިނިޓު reads ތިން ގަޑި ތިރީސް:ހަތަރު މިނިޓު.
• Percentages: negative percents, the fraction separator in comma-decimal mode, and 50% އަށްވުރެ (no dative inflection) are read more literally than a speaker would.
• Units such as kg, km, GB have no Dhivehi lexicon yet and follow unknown_latin; Roman numerals are spelled letter by letter.
• English words inside Dhivehi sentences are left as written by default (unknown_latin="passthrough"); there is no transliteration.
• Score draws like 2-2 use the linking form ދޭދޭ; other scores read both numbers in full (3-2 → ތިނެއް ދޭއް).
• Arabic-script quotations (Quran, hadith) pass through untouched.

Report a wrong reading with the written sentence and the form you would say.

Configuration

NormalizerConfig(
    dialect="spoken",            # only option for now
    unknown_latin="passthrough", # "passthrough" | "drop" | "spell"
    decimal_separator="auto",    # "auto" | "dot" | "comma"
    time_system="auto",          # "auto" | "12" | "24"
    currency_default="MVR",
    keep_punctuation=True,
    diagnostic=False,
    strict=False,
)

• dialect="spoken" — the only option today; reserved for a future literary/written-register mode.
• unknown_latin — controls plain Latin words only, never acronyms and never URLs: 2-4 letter all-caps runs (MNDF) are always spelled via Thaana letter names regardless of this setting, and URLs/e-mails/handles/ hashtags always pass through untouched regardless of this setting (even "drop" never removes a URL — only ordinary Latin words like STORE). "passthrough" (default) leaves other Latin words unchanged; "drop" removes them (a Latin-only input can yield an empty string); "spell" spells them out letter-by-letter.
• decimal_separator — "auto" and "dot" both use . as the decimal mark and ,/، as thousands separators. "comma" swaps that: , is the decimal mark (1,5 → އެކެއް ޕޮއިންޓް ފަހެއް), except a chain of ≥2 three-digit , groups is still read as thousands grouping (1,234,567 stays one cardinal, not a decimal).
• time_system — "auto" and "24" read the hour as-is (14:30 → ސާދަ ގަޑި ތިރީސް). "12" prefixes a day-period word and reduces the hour to a 12-hour clock value (14:30 → މެންދުރުފަހު ދެ ގަޑި ތިރީސް).
• currency_default — documented as reserved; only "MVR" (ރ) is wired today, so changing it currently has no effect.
• keep_punctuation — when False, punctuation tokens are dropped from the output instead of being kept verbatim.
• diagnostic — internal flag used by Normalizer.trace(); leave this False when calling normalize()/Normalizer.__call__ directly.
• strict — when True, raises ValueError if any ASCII or Arabic-Indic digit survives in the final output instead of silently passing it through. Digits inside a passed-through URL/e-mail/handle token are exempt from this check — strict=True does not raise on https://example.com/12345, since that digit was never meant to be read aloud in the first place.

Diagnostic mode

Normalizer.trace(text) returns the classified token list instead of joined text. Useful for debugging which rule fired:

for tok in Normalizer().trace("ޑރ. އިބްރާހިމް 2024ގައި"):
    print(tok.cls, tok.text, tok.spoken, tok.fields)

License

MIT — see LICENSE.