phonemizer 2.0.1

Simple text to phonemes converter for multiple languages

Phonemizer -- foʊnmaɪzɚ

• Simple text to phonemes converter for multiple languages, based on festival, espeak-ng and segments.

• Provides both the phonemize command-line tool and the Python function phonemizer.phonemize

• espeak-ng is a text-to-speech software supporting multiple languages and IPA (Internatinal Phonetic Alphabet) output. See https://github.com/espeak-ng/espeak-ng. Alternatively you can use the orginal espeak program (espeak-ng is a fork of espeak supporting much more languages and significant improvements).

• festival is also a text-to-speech software. Currently only American English is supported and festival uses a custom phoneset (http://www.festvox.org/bsv/c4711.html), but festival is the only backend supporting tokenization at the syllable level. See http://www.cstr.ed.ac.uk/projects/festival.

• segments is a Unicode tokenizer that build a phonemization from a grapheme to phoneme mapping provided as a file by the user. See https://github.com/cldf/segments.

Installation

You need python>=3.6. If you really need to use python2, use an older version of phonemizer.

Dependencies

• You need to install festival and espeak-ng on your system. Visit this festival link and that espeak-ng one for installation guidelines. On Debian/Ubuntu simply run:

    $ sudo apt-get install festival espeak-ng
  

• Alternatively you may want to use espeak instead of espeak-ng, see here for instalaltion instructions.

Phonemizer

• The simplest way is using pip:

    $ pip install phonemizer
  

• OR install it from sources with:

    $ git clone https://github.com/bootphon/phonemizer
    $ cd phonemizer
    $ python setup.py build
    $ [sudo] python setup.py install
  

  If you experiment an error such as ImportError: No module named setuptools during installation, refeer to issue 11.

Docker image

Alternatively you can run the phonemizer within docker, using the provided Dockerfile. To build the docker image, have a:

$ git clone https://github.com/bootphon/phonemizer
$ cd phonemizer
$ sudo docker build -t phonemizer .

Then run an interactive session with:

$ sudo docker run -it phonemizer /bin/bash

Command-line examples

For a complete list of available options, have a:

$ phonemize --help

See the installed backends with the --version option:

$ phonemize --version
phonemizer-2.0
available backends: festival-2.5.0, espeak-ng-1.49.3, segments-2.0.1

Input/output exemples

• from stdin to stdout:

    $ echo "hello world" | phonemize
    həloʊ wɜːld
  

• from file to stdout

    $ echo "hello world" > hello.txt
    $ phonemize hello.txt
    həloʊ wɜːld
  

• from file to file

    $ phonemize hello.txt -o hello.phon --strip
    $ cat hello.phon
    həloʊ wɜːld
  

Token separators

You can specify separators for phonemes, syllables (festival only) and words.

$ echo "hello world" | phonemize -b festival -w ' ' -p ''
hhaxlow werld

$ echo "hello world" | phonemize -b festival -p ' ' -w ''
hh ax l ow w er l d

$ echo "hello world" | phonemize -b festival -p '-' -s '|'
hh-ax-l-|ow-| w-er-l-d-|

$ echo "hello world" | phonemize -b festival -p '-' -s '|' --strip
hh-ax-l|ow w-er-l-d

$ echo "hello world" | phonemize -b festival -p ' ' -s ';esyll ' -w ';eword '
hh ax l ;esyll ow ;esyll ;eword w er l d ;esyll ;eword

You cannot specify the same separator for several tokens (for instance a space for both phones and words):

$ echo "hello world" | phonemize -b festival -p ' ' -w ' '
fatal error: illegal separator with word=" ", syllable="" and phone=" ",
must be all differents if not empty

Options

• Espeak us-english is the default

    $ echo "hello world" | phonemize
    həloʊ wɜːld
    $ echo "hello world" | phonemize -l en-us -b espeak
    həloʊ wɜːld
  

• use Festival US English instead

    $ echo "hello world" | phonemize -l en-us -b festival
    hhaxlow werld
  

• In French, using espeak

    $ echo "bonjour le monde" | phonemize -b espeak -l fr-fr
    bɔ̃ʒuʁ lə- mɔ̃d
  
    $ echo "bonjour le monde" | phonemize -b espeak -l fr-fr -p ' ' -w ';eword '
    b ɔ̃ ʒ u ʁ ;eword l ə- ;eword m ɔ̃ d ;eword
  

• In Japanese, using segments

    $ echo 'konnichiwa' | phonemize -b segments -l japanese
    konnitʃiwa
  
    $ echo 'konnichiwa' | phonemize -b segments -l ./phonemizer/share/japanese.g2p
    konnitʃiwa
  

• Espeak can output SAMPA phonemes instead of IPA ones (this is only supported by espeak-ng, not by the original espeak)

    $ echo "hello world" | phonemize -l en-us -b espeak --sampa
    h@loU w3:ld
  

• Espeak can output the stresses on phonemes (this is not supported by festival or segments backends)

    $ echo "hello world" | phonemize -l en-us -b espeak --with-stress
    həlˈoʊ wˈɜːld
  

• Espeak can switch languages during phonemization (below from French to English), use the --language-switch option to deal with it

    $ echo "j'aime le football" | phonemize -l fr-fr -b espeak --language-switch keep-flags
    [WARNING] fount 1 utterances containing language switches on lines 1
    [WARNING] extra phones may appear in the "fr-fr" phoneset
    [WARNING] language switch flags have been kept (applying "keep-flags" policy)
    ʒɛm lə- (en)fʊtbɔːl(fr)
  
    $ echo "j'aime le football" | phonemize -l fr-fr -b espeak --language-switch remove-flags
    [WARNING] fount 1 utterances containing language switches on lines 1
    [WARNING] extra phones may appear in the "fr-fr" phoneset
    [WARNING] language switch flags have been removed (applying "remove-flags" policy)
    ʒɛm lə- fʊtbɔːl
  
    $ echo "j'aime le football" | phonemize -l fr-fr -b espeak --language-switch remove-utterance
    [WARNING] removed 1 utterances containing language switches (applying "remove-utterance" policy)
  

Supported languages

• Languages supported by festival are:

    en-us	->	english-us
  

• Languages supported by the segments backend are:

    chintang  -> ./phonemizer/share/chintang.g2p
    cree	  -> ./phonemizer/share/cree.g2p
    inuktitut -> ./phonemizer/share/inuktitut.g2p
    japanese  -> ./phonemizer/share/japanese.g2p
    sesotho	  -> ./phonemizer/share/sesotho.g2p
    yucatec	  -> ./phonemizer/share/yucatec.g2p
  

  Instead of a language you can also provide a file specifying a grapheme to phoneme mapping (see the files above for exemples).

• Languages supported by espeak are (espeak-ng supports even more of them), type phonemize --help for an exhaustive list:

    af	->	afrikaans
    an	->	aragonese
    bg	->	bulgarian
    bs	->	bosnian
    ca	->	catalan
    cs	->	czech
    cy	->	welsh
    da	->	danish
    de	->	german
    el	->	greek
    en	->	default
    en-gb	->	english
    en-sc	->	en-scottish
    en-uk-north	->	english-north
    en-uk-rp	->	english_rp
    en-uk-wmids	->	english_wmids
    en-us	->	english-us
    en-wi	->	en-westindies
    eo	->	esperanto
    es	->	spanish
    es-la	->	spanish-latin-am
    et	->	estonian
    fa	->	persian
    fa-pin	->	persian-pinglish
    fi	->	finnish
    fr-be	->	french-Belgium
    fr-fr	->	french
    ga	->	irish-gaeilge
    grc	->	greek-ancient
    hi	->	hindi
    hr	->	croatian
    hu	->	hungarian
    hy	->	armenian
    hy-west	->	armenian-west
    id	->	indonesian
    is	->	icelandic
    it	->	italian
    jbo	->	lojban
    ka	->	georgian
    kn	->	kannada
    ku	->	kurdish
    la	->	latin
    lfn	->	lingua_franca_nova
    lt	->	lithuanian
    lv	->	latvian
    mk	->	macedonian
    ml	->	malayalam
    ms	->	malay
    ne	->	nepali
    nl	->	dutch
    no	->	norwegian
    pa	->	punjabi
    pl	->	polish
    pt-br	->	brazil
    pt-pt	->	portugal
    ro	->	romanian
    ru	->	russian
    sk	->	slovak
    sq	->	albanian
    sr	->	serbian
    sv	->	swedish
    sw	->	swahili-test
    ta	->	tamil
    tr	->	turkish
    vi	->	vietnam
    vi-hue	->	vietnam_hue
    vi-sgn	->	vietnam_sgn
    zh	->	Mandarin
    zh-yue	->	cantonese
  

Licence

Copyright 2015-2019 Mathieu Bernard

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.