Skip to main content


Project description

# Multicode: Miscelleneous functions for handling unicode in transcriptions

[![Build Status](]( [![codecov](]( [![PyPI](](

## General information

This repository provides code and data to normalize datasets, to avoid pitfalls of unicode when creating linguistic data, and to infer general information about digital data when dealing with it in linguistic contexts.

One major concern is the normalization of phonetic transcriptions. There are many lookalikes in unicode which linguists use without knowing that they are not what they initially want to write. Thus the character used to transcribe an alveolar voiceless fricative, for example, has three candidates in Unicode which look extremely similar: `ʃ ∫ ꭍ`, although only the first one is the character that should be used when writing phonetic transcriptions. Our goal is to provide ways to detect and normalize those cases which result from the confusion of Unicode characters when transcribing languages phonetically.

Other cases include specific writing systems, like CJK systems, where we want to offer services to more quickly handle with them. These cases include the possibility to detect the Unicodes of very rare characters by naming only there parts, but also simple Python functions that allow to determine whether a given character is a Chinese character or not.

## Structure of the Repository

Our repository contains data and code. Data is presented in one of two basic formats. We use YAML for the convenient storing of confusable characters, and for the translation among different transcription systems (such as sound classes in the LingPy library), and TSV for tabular data. We prefer YAML over JSON for reasons of readability.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for multicode, version 0.1.0
Filename, size File type Python version Upload date Hashes
Filename, size multicode-0.1.0.tar.gz (16.8 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page