AGILe: Ancient Greek Inscriptions Lemmatizer

AGILe is a lemmatizer for Ancient Greek inscriptions developed at the University of Groningen. Details can be found in:

de Graaf, E., Stopponi, S., Bos, J., Peels-Matthey, S. & Nissim, M. (2022). AGILe: The First Lemmatizer for Ancient Greek Inscriptions. Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), Marseille, 20-25 June 2022. pp. 5334–5344. http://www.lrec-conf.org/proceedings/lrec2022/pdf/2022.lrec-1.571.pdf

Peels-Matthey, S., de Graaf, E., Nissim, M., Bos, J. & Stopponi, S. (2024). Automatic lemmatization of ancient Greek inscriptions: A presentation of AGILe. Journal of epigraphic studies. 7, 2024: 29-50. https://pure.rug.nl/ws/portalfiles/portal/1054237044/Peels-Matthey_et_al_2024_Automatic_lemmatization_of_Ancient_Greek_inscriptions_-_A_presentationof_AGILe.pdf

Relation to agile-gronlp/agile

This repository is a maintained fork of agile-gronlp/agile, packaged for installation with pip. The differences that matter to you:

• No cltk dependency. The original required cltk 1.0.21, which no longer installs on recent Python versions (its pinned PyYAML 5.4.1 has no wheels past Python 3.9 and does not build against Cython 3). Only one function was needed, and it now lives in this package.
• Installable from PyPI. No cloning, no manual dependency wrangling.
• Runs on Windows, macOS and Linux.

If you are following instructions that tell you to clone agile-gronlp/agile and run pip install -r requirements.txt, you are on the old path and will hit a build error on PyYAML. Use the instructions below instead.

Installation

pip install ancientgreek

Or straight from this repository:

pip install git+https://github.com/prhbrt/ancientgreek

Python 3.9 or later. Use version 0.2.2 or newer — earlier releases did not declare their dependencies and install into a state where import ancientgreek fails.

Installing into a virtual environment is strongly recommended, because of the next section.

A note on size

AGILe uses Stanza, which is built on PyTorch. On Linux, pip installs the CUDA-enabled build of PyTorch by default, which pulls in the NVIDIA runtime libraries and comes to roughly 2.5 GB — whether or not your machine has a GPU.

If you are running on a laptop or any machine without an NVIDIA GPU, install the CPU-only build of PyTorch first and the rest will follow it:

pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install ancientgreek

This cannot be expressed in the package metadata, which is why it is a manual step.

Downloading the Stanza models

The Ancient Greek models from Stanza are downloaded once, from your Python interpreter:

>>> import stanza
>>> stanza.download('grc')

Usage

>>> from ancientgreek import lemmatize

>>> doc = lemmatize("αἲξ θύεται τάδε μὴ ἐσφέρεν ἐς τὸ τέμενος τοῦ Ἀπόλλωνος τοῦ Οὐλίου εἱμάτιον")
>>> for sent in doc.sentences:
...     for word in sent.words:
...         print(f'word: {word.text + " ":15}lemma: {word.lemma}')

This gives:

word: αἲξ            lemma: αἴξ
word: θύεται         lemma: θύω
word: τάδε           lemma: ὅδε
word: μὴ             lemma: μή
word: ἐσφέρεν        lemma: εἰσφέρω
word: ἐς             lemma: εἰς
word: τὸ             lemma: τε
word: τέμενος        lemma: τέμενος
word: τοῦ            lemma: ποῦ
word: Ἀπόλλωνος      lemma: Ἀπόλλων
word: τοῦ            lemma: ποῦ
word: Οὐλίου         lemma: οὔλιος
word: εἱμάτιον       lemma: ἱμάτιον

The lexicon lookup can be disabled by setting the use_lexicon parameter of lemmatize to False.

Each line of the text is treated as one sentence; sentence splitting is not supported.

Command line

Installing the package also installs an agile command, which lemmatizes inscriptions from files, folders or standard input without writing any Python:

agile inscription.txt

The output is tab separated by default, so it opens straight into Excel or LibreOffice:

document         sentence  id  text     lemma
inscription.txt  1         1   αἲξ      αἴξ
inscription.txt  1         2   θύεται   θύω
inscription.txt  1         3   τάδε     ὅδε

Inputs are files, folders (searched recursively for *.txt), or - for standard input, which is also what is read when no input is given at all:

echo "αἲξ θύεται τάδε" | agile
agile corpus/                          # every .txt under corpus/, recursively
agile attica.txt ionia.txt

Output goes to standard output, to a single file, or to a folder mirroring the input folder:

agile corpus/ -o lemmas.tsv            # everything in one file
agile corpus/ -d lemmas/               # one output file per input file

The model is loaded once per run, so lemmatizing a whole folder in one command is far faster than running the command once per file.

option	what it does
-o FILE	write everything to one file instead of standard output
-d DIR	write one file per input file, mirroring the folder structure
-f FORMAT	tsv (default), jsonl (with character offsets), conllu, or txt for the lemmas only
--per-line	treat every line of an input as a separate inscription
--no-lexicon	do not correct the predicted lemmas against the lexicon
--keep-prediction	also report the uncorrected model output as lemma_pred
--glob PATTERN	which files to take from a folder, *.txt by default
--model REPO_OR_PT	a different huggingface repository, or a local .pt model
-q	do not report progress on standard error

agile --help lists them all.

To check your installation from the shell:

agile --version
echo "αἲξ θύεται τάδε" | agile

On Windows, set PYTHONUTF8=1 first, otherwise printing Greek to the console raises a UnicodeEncodeError:

$env:PYTHONUTF8=1

Interactive notebook on Google Colab

To try AGILe without installing it: https://colab.research.google.com/drive/1YZMGxF8ORCrk_tyD1muHkgVsMXxeWHJJ?usp=drive_link

Acknowledgements

The lexicon.p used is extracted from an XML edition with composed Unicode of the LSJ, as transformed by Giuseppe G. A. Celano. The original text is provided under a CC BY-SA license by Perseus Digital Library, http://www.perseus.tufts.edu, with funding from The National Endowment for the Humanities. Data accessed from https://github.com/PerseusDL/lexica/.

License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

BibTeX

@InProceedings{degraaf-EtAl:2022:LREC,
  author    = {de Graaf, Evelien  and  Stopponi, Silvia  and  Bos, Jasper K.  and  Peels-Matthey, Saskia  and  Nissim, Malvina},
  title     = {AGILe: The First Lemmatizer for Ancient Greek Inscriptions},
  booktitle = {Proceedings of the Language Resources and Evaluation Conference},
  month     = {June},
  year      = {2022},
  address   = {Marseille, France},
  publisher = {European Language Resources Association},
  pages     = {5334--5344},
  url       = {https://aclanthology.org/2022.lrec-1.571}
}

@article{peels2024automatic,
  title={Automatic lemmatization of ancient Greek inscriptions: A presentation of AGILe},
  author={Peels-Matthey, Saskia and de Graaf, Evelien and Nissim, Malvina and Bos, Jasper and Stopponi, Silvia},
  journal={Journal of epigraphic studies: 7, 2024},
  pages={29--50},
  year={2024},
  publisher={Fabrizio Serra}
}