graphtransliterator 0.2.1

A graph-based transliteration tool

Graph Transliterator

A graph-based transliteration tool that lets you convert the symbols of one language or script to those of another using rules that you define.

Graph Transliterator can be used to:

• convert language alphabets to transliteration and vice versa

• convert between transliteration systems

• convert legacy fonts and encodings into Unicode

• Free software: MIT license

• Documentation: https://graphtransliterator.readthedocs.io.

• Requires: Python 3.5, 3.6, or 3.7

Features

• Provides a transliteration tool that can be configured to convert the tokens of an input string into an output string using:

  • definitions of input tokens and token classes

  • transliteration rules based on:

    • a sequence of input tokens

    • specific input tokens that precede or follow the token sequence

    • classes of input tokens preceding or following specific tokens

  • “on match” rules for output to be inserted between transliteration rules involving particular token classes

  • defined rules for whitespace, including its optional consolidation

• Can be setup using:

  • an “easy reading” YAML format that lets you quickly craft settings for the transliteration tool

  • “direct” settings using a dictionary

• Automatically orders rules based on specificity using the number of tokens in a transliteration rule

• Checks for ambiguity in transliteration rules, where two rules of equal specificity could match the same tokens

• Provides details about each transliteration rule matched

• Allows optional matching of all possible rules in a particular location

• Permits pruning of rules with certain productions

• Can be serialized as a dictionary for export to JSON, etc.

• Provides full support for Unicode, including Unicode character names in the “easy reading” YAML format

• Constructs and uses a directed tree and performs a best-first search to find the most specific transliteration rule in a given context

History

[Unreleased - Maybe]

• Add CLI.

• Add metadata guidelines

• Save match location in tokenize

• Reconsider serialization keys

• Add tests directly to YAML files

• Allow insertion of transliteration error messages into output.

[Unreleased-TODO]

0.2.1 (2019-07-23)

• fixed HISTORY.RST for PyPI

0.2.0 (2019-07-23)

• Fixed module naming in docs using __module__.

• Converted DirectedGraph nodes to a list.

• Added Code of Conduct.

• Added GraphTransliterator class.

• Updated module dependencies.

• Added requirements.txt

• Added check_settings parameter to skip validating settings.

• Added tests for ambiguity and check_ambiguity parameter.

• Changed name to Graph Transliterator in docs.

• Created core.py, validate.py, process.py, rules.py, initialize.py, exceptions.py, graphs.py

• Added ignore_errors property and setter for transliteration exceptions (UnrecognizableInputToken, NoMatchingTransliterationRule)

• Added logging to graphtransliterator

• Added positive cost function based on number of matched tokens in rule

• added metadata field

• added documentation

0.1.1 (2019-05-30)

• Adjust copyright in docs.

• Removed Python 2 support.

0.1.0 (2019-05-30)

• First release on PyPI.