Skip to main content

A spelling similarity measure for cognate identification.

Project description

spsim is a Python 3 module that implements a spelling similarity measure for identifying cognates across languages, taking into account spelling differences that are characteristic of each language pair, as described in [Gomes2011].

Note: in the examples below, $ denotes the Bash prompt and a Linux, MacOs or similar *nix environment is assumed.

Install as usual:

$ pip3 install spsim

Example command line usage:

$ # first let's get some pairs of words that may be cognates:
$ wget http://research.variancia.com/spsim/maybe_enpt.txt
$ cat maybe_enpt.txt
pharmacy    farmácia
arithmetic  aritmética

$ # If we don't give any example cognates, SpSim will be equivalent to
$ #             1 - edit_distance / max_len_of_strings
$ # Note that by default spsim matches accentuated characters, i.e. a == á
$ echo "" > empty.txt
$ spsim empty.txt maybe_enpt.txt
pharmacy    farmácia    0.5
arithmetic  aritmética  0.8

$ now let's get some example cognates:
$ wget http://research.variancia.com/spsim/examples_enpt.txt
$ cat examples_enpt.txt
alcohol     álcool
alpha       alfa
anomaly     anomalia
mathematics matemática
methodology metodologia
metric      métrica
morphine    morfina
photos      fotos

$ # by giving these examples to spsim, it will learn to ignore certain differences:
$ spsim examples_enpt.txt maybe_enpt.txt
pharmacy    farmácia    1.0
arithmetic  aritmética  1.0
[Gomes2011]Measuring Spelling Similarity for Cognate Identification, Luís Gomes and Gabriel Pereira Lopes in Progress in Artificial Intelligence, 15th Portuguese Conference in Artificial Intelligence, EPIA 2011, Lisboa, Portugal, October 2011, http://www.springerlink.com/content/gtl56j3l06906020/

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
spsim-0.1.2-py3-none-any.whl (8.6 kB) Copy SHA256 hash SHA256 Wheel py3 Jul 26, 2017
spsim-0.1.2.tar.gz (6.0 kB) Copy SHA256 hash SHA256 Source None Jul 26, 2017

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page