How does it work?

re_transliterate relies on Python dictionaries - mappings of x:y. These mappings should be written as regular expressions, which you may already be familiar with. If not, find a computer science undergrad and make them write your mappings :)

In simple cases, you can simply use normal strings (prefaced by ‘u’ for Unicode in Python 2):

>>> consonant_map = {u"ch":u"č", u"x’":u"š’"}
>>> word_replace(consonant_map, u"ch")
č

With that mapping, occurrences of “ch” will be turned into ‘č’ and ejectives written with ‘x’ will instead use ‘š’.

The power of regular expressions

In more complicated cases, allowing regular expressions makes life much simpler. We can use character classes to reduce repetition, and use backslashes to do fancy things. If your string contains a backslash, preface it with ‘r’ as well as ‘u’. An example of both:

>>> VOWELS = u"([aeiouáéíóú])"
>>> long_vowels = {VOWELS + u":":ur"\g<1>ː"}
>>> word_replace(long_vowels, u"a:'")
aː

With this mapping, vowels followed by a colon character (‘:’) will be changed to use the IPA symbol for length (‘ː’). Here, square brackets [] indicate a “character class”: these characters will be treated in the same way, such that the string “a:” will be matched just like “i:” or “ú:”.

Then, the parentheses () indicate a “group” in the regular expression: this allows us to reference what it is that the pattern matched - that’s what the g<1> does, referring to “group 1”. If the pattern were to match “a:”, it would insert “aː”. This is the primary benefit of using regular expressions: we don’t have to specify that “a:” turns into “aː”, and that “e:” turns into “eː”, etc.

Complicated mappings

In the previous example, did you see that I used addition inside the mapping? This allows useful patterns and special Unicode sequences to be saved globally and reused elsewhere. In my own work, I defined the above character class of vowels, and the Unicode sequence for the “tilde below letter” diacritic:

>>> LARYNGEAL = ur"\u0330"

This allowed me to write mappings like this, where the ‘ character represents laryngealization and may occur before/after a marker for length:

>>> long_laryngealized = {VOWELS + u"(:'|':)":
                          ur"\g<1>" + LARYNGEAL + u"ː"}

This converts any vowel, followed by either (marked by ‘|’) “:’” or “’:”, into the vowel + laryngealization diacritic + length marker.

Applying your mappings

Once you’ve created all your mappings, you can then either apply them one at a time to a word (word_replace) or list of words (word_list_replace). As you develop your conversion code, you should use these functions first, to make sure everything is working correctly.

Generally, you’ll want to have multiple mappings, because some conversions will be sensitive to ordering. For my work, I created three mappings: trigraphs (sequences of three characters), bigraphs, and monographs. I then applied each of those in order, using the two transliterate functions. Example:

>>> re_trigraphs = {u"lh’":u"ɬ’",
                    VOWELS + u":'|':":ur"\g<1>" + LARYNGEAL + u"ː"}
>>> re_bigraphs = {u"ch":u"č", u"lh":u"ɬ", u"nh":u"ŋʔ",
                   u"tz":u"c", u"uj":u"ʍ",
                   u"s’":u"s’", u"x’":u"š’",
                   VOWELS + u":":ur"\g<1>ː",
                   VOWELS + u"'":ur"\g<1>" + LARYNGEAL}
>>> re_monographs = {u"h":u"ʔ", u"r":u"ɾ", u"x":u"š"}
>>> order = [re_trigraphs, re_bigraphs, re_monographs]
>>> transliterate(order, u"a:ma'ha:'pi'tzín")
aːma̰ʔa̰ːpḭcín

This way, the conversion for “lh”:”ɬ” won’t interrupt the conversion of “lh’”:u”ɬ’”. Alternatively, I suppose you could use an OrderedDict with the word_replace/word_list_replace functions, but I personally like it this way.

Conclusion

If you’re writing your code in an interactive environment such as IPython, note that if you change the contents of a dictionary, it will not be updated inside any lists you may have created. I learned this the hard way, when my code didn’t work until I redefined the order list used above. If you see any weird bugs, that might be the cause.

Other than that, if you have any other questions/issues with the module, shoot me an e-mail; especially if you run into any issues with Unicode support or bugs specific to Python 3/PyPy/another interpreter.

For an example transliteration script, see the script I wrote for Upper Necaxa Totonac. The example usage above is all drawn from that code - but it might help to see a working example.

======== Versions ========

1.3

Debugged readme examples (Unicode escape was wrong, and | was used incorrectly)

1.2

Readme should be perfect

1.1

Fixing readme for PyPI

1.0

Factored out of the UNT_to_IPA module