betacode 1.1

betacode

Convert betacode to unicode and vice-versa easily. The definition used is based off what is found at the TLG Beta Code Manual. Only the Greek sections were paid attention to.

Motivation

I was working a classics research project and had to use the Perseus catalog to extract some Greek work. Much to my surprise however, the only download I could find was a betacode version. An encoding that is over 30 years old, rather than modern, fancy, clean unicode. There was no nice pip package that I could easily go to for this simple task, so I decided to roll my own.

Install

Installation is easy. Use pip or your preferred method to download from PyPI.

pip install betacode

Usage

Note that in all examples, strings are unicode encoded. Input can be in upper or lower case. The official definition from TLG uses only uppercase, but many resources, such as the Perseus catalog, are encoded in lowercase, so this package accepts both. This package also can disregard the unnecessary cannonical order of Greek diacritics from the official definition. The only thing that matters in order for the betacode to be unambiguous is that each unit must either begin with a * or a letter. As long as these constraints are followed, breathing marks, accents, and such can go in any order. However, the cannonical order will be returned when going from unicode to betacode. Also note that currently, only individual, non-combining characters are handled. This means that you cannot do all combinations of letters and diacritics. Only those defined as composite characters in the Greek and Extended Greek sections of unicode.

Betacode to unicode

import betacode

beta = 'analabo/ntes de\ kaq\' e(/kaston'
betacode.beta_to_uni(beta) # αναλαβόντες δὲ καθ᾽ ἕκαστον

Note that polytonic accent marks will be used, and not monotonic accent marks. Both are de jure equivalent in Greece, but betacode was initially developed to encode classic works so the polytonic diacritics are more fitting. In other words, the oxeîa will be used rather than tónos. The oxeîa form can be converted to the modern accent form easily either through search and replace, or unicode normalization since oxeîa has canonical decomposition into tónos.

Conversion can also be made more strict by using the strict flag.

beta_to_uni(text, strict=False)

If set, only the cannonical order of diacritics is accepted in betacode. If it is not set, then any order is allowed as long as capital letters begin with a * and lowercase letters begin with the letter and not a diacritic.

Unicode to betacode

import betacode

uni = 'αναλαβόντες δὲ καθ᾽ ἕκαστον'
betacode.uni_to_beta(uni) # analabo/ntes de\ kaq\' e(/kaston

The unicode text can use polytonic (oxeîa) accent marks or monotonic (tónos) accent marks can be used.

Command line usage

Installing the package also installs a betacode command with two subcommands, to-unicode and to-beta. Each accepts one of -t/--text for raw text on the command line, -f/--file for a file to convert, or -i/--interactive for a continuous, REPL-like session.

Convert raw text given directly on the command line:

betacode to-unicode --text "lo/gos"
# λόγος

betacode to-beta --text "λόγος"
# lo/gos

Add --strict to to-unicode to only accept the canonical diacritic order:

betacode to-unicode --strict --text "lo/gos"

Convert the contents of a file, writing the result to stdout:

betacode to-unicode --file input.txt > output.txt

Start a continuous session, where each line entered is converted and printed until you exit with Ctrl-D:

$ betacode to-unicode --interactive
Entering continuous mode. Press Ctrl-D (or Ctrl-Z on Windows) to exit.
>> lo/gos
λόγος
>> kalo/s
καλός
>>

Speed

The original implementation used a custom made trie. This maybe was not the fastest (I wasn't sure). So, I compared against a third party trie implementation, pygtrie. The pygtrie had nicer prefix methods which allowed for much faster processing of large texts. This changed converting all of Strabo or Herodotus in the Perseus catalog from a many minute operation to a ~3-4 second operation. I have seen implementations that use regular expressions which I suspsect might be faster since the underlying implementation is in C. However, this package is much smaller and simpler if betacode conversion is all that is needed than CLTK, for example.

Modified Betacode

There is talk of a modified betacode that I have seen around on the internet. I have never been able to find a definitive definition of this so I have not implemented it. Among some differences is word final sigma usage, _ as macron, and uppercase and lowercase roman letters instead of using *.

Development

I am no classicist, and this was done in my free time. It is very possible that there are some letters missing that are not accounted for, or some punctuation that is not properly handled. If that is the case, please tell me as it is easy to fix, or please open a PR for your own branch. Write tests if you do add a feature.