nuficlean 0.1.0

Nufi language text normalisation: Bana → Komako standard orthography, tone stripping, mojibake repair

nuficlean

Python library for normalising Nufi (Babanki-Tungo / Nʉ Fì) text: converts Bana orthography to Komako standard, strips low-tone diacritics, and repairs common encoding issues.

Install

pip install nuficlean

Or from source:

pip install -e /path/to/nuficlean

Usage

from nuficlean import clean

clean("kòlə̀'")        # → "kwele'"
clean("tōh mēndɑ̀'")  # → "tōh mēndɑ'"
clean("mbɑ̀ɑ̀")        # → "mbɑɑ"

Batch cleaning:

from nuficlean import clean_lines

clean_lines(["kòlə̀'", "mbɑ̀ɑ̀"])  # → ["kwele'", "mbɑɑ"]

CLI:

nuficlean "kòlə̀'"
echo "mbɑ̀ɑ̀" | nuficlean

Pipeline

1. Mojibake repair — fixes Latin-1 → UTF-8 misencoding
2. Apostrophe / quote unification — maps ', `, ʼ, ", «, », etc. to ASCII
3. Bana → Komako rewrite — longest-match substitution table (e.g. kòlə̀' → kwèlè', ɛ̀ → a)
4. Low-tone stripping — removes grave-accent tone marks (à→a, ɑ̀→ɑ, …)
5. NFC recomposition

License

MIT