Mirandese Phonemizer

This repository contains a Python-based Mirandese phonemizer, designed to convert Mirandese text into its International Phonetic Alphabet (IPA) representation. It supports different Mirandese dialects and incorporates various phonological rules based on linguistic information from Wikipedia.

Features

• Grapheme-to-Phoneme Conversion: Converts Mirandese graphemes (letters and digraphs) to their corresponding IPA phonemes.
• Contextual Rules: Applies phonological rules based on surrounding characters (e.g., lenition of voiced stops, sibilant variations, vowel glides).
• Dialectal Support:
  • Handles the specific lh to l pronunciation change in the Sendinese dialect.
  • Includes word-level lookup dictionaries for Central, Raiano, and Sendinese dialects to handle irregular pronunciations.
• Latin Cluster Evolution: Converts Latin initial consonant clusters (pl, kl, fl) to /tʃ/.
• Proto-Romance Medial Clusters: Converts Proto-Romance medial clusters (-ly-, -cl-) to /ʎ/.
• Double Consonant Palatalization: Handles palatalization of double ll to /ʎ/ and nn to /ɲ/.
• Proto-Romance -mn-: Converts -mn- to /m/.
• Output Customization: Options to keep or remove optional phonemes (in parentheses) and stress marks/syllable dots.

Architecture

The rule-based engine (OrthographyRulesMWL) is built on the shared orthography2ipa pronunciation lattice. Grapheme segmentation is delegated to the language-agnostic orthography2ipa.phonetok.PhonetokTokenizer (a maximal-munch trie over the Mirandese grapheme set declared in g2p.json) and every context-sensitive realisation rule (lenition, sibilant voicing, palatalisation, i/u glides, final -o raising, the trill/tap r split) runs as an orthography2ipa.rescorer.LatticeRescorer over the resulting per-grapheme SegmentSlot lattice. There is no private tokenizer and no hand-rolled index arithmetic — the engine shares the same segmentation substrate as every other orthography2ipa downstream (Barranquenho, arbtok, tugaphone).

Usage

from mwl_phonemizer import CRFOrthoCorrector


sample_texts = [
    "Muitas lhénguas ténen proua de ls sous pergaminos antigos, de la lhiteratura screbida hai cientos d'anhos i de scritores hai muito afamados, hoije bandeiras dessas lhénguas. Mas outras hai que nun puoden tener proua de nada desso, cumo ye l causo de la lhéngua mirandesa.",
    "Todos ls seres houmanos nácen lhibres i eiguales an honra i an dreitos. Dotados de rezon i de cuncéncia, dében de se dar bien uns culs outros i cumo armano",
]

phonemizer = CRFOrthoCorrector()
for text in sample_texts:
    print(f"Original: {text}")
    print(f"Phonemized: {phonemizer.phonemize_sentence(text)}\n")

Helper Functions

The base class provides static methods for cleaning up IPA output:

• strip_markers(ipa: str): Removes syllable dots (.) and optional phoneme parentheses (()).
• strip_stress(ipa: str): Removes primary (ˈ) and secondary (ˌ) stress markers.

ipa_with_markers = "ˈe(j).ʒɛmˈplu"
clean_ipa = phonemizer.strip_markers(ipa_with_markers)
print(f"Clean IPA: {clean_ipa}")
# Output: Clean IPA: ˈejʒɛmˈplu

ipa_with_stress = "miɾɐ̃ˈdes̺"
stress_agnostic_ipa = phonemizer.strip_stress(ipa_with_stress)
print(f"Stress-Agnostic IPA: {stress_agnostic_ipa}")
# Output: Stress-Agnostic IPA: miɾɐ̃des̺

Phonemizer Comparison

Notes:

• For Epitran and Espeak, the first value is the initial phonemization output; the second is after applying correction rules.
• PER (Phoneme Error Rate) measures the proportion of phonemes differing from the gold standard IPA transcription.
• Stress-Agnostic PER ignores stress marks.
• lower PER does not necessarily mean a better phonemizer
• CRF models are overfitted due to small data size

Future Work

• Stress Prediction: Implement machine learning or rule-based stress placement.
• Larger Dataset: Expand training and evaluation beyond ~150 words.
• Neural Approaches: Explore Transformer or seq2seq models for improved G2P accuracy.
• Dialectal Coverage: Add more irregular forms and dialect-specific exceptions.

Contributing

Feel free to open issues or submit pull requests for improvements, bug fixes, or new dialect support.