Joey NMT

Joey NMT is a minimalist neural machine translation toolkit for educational purposes.

It aims to be a clean and minimalistic code base to help novices pursuing the understanding of neural machine translation.

Installation

Joey NMT is built on PyTorch. Please make sure you have a compatible environment. We tested Joey NMT v2.3 with

• python 3.11
• torch 2.1.2
• cuda 12.1

Warning
When running on GPU, you need to manually install the suitable PyTorch version for your CUDA version. See PyTorch installation instructions.

You can install Joey NMT either A. via pip or B. from source.

A. Via pip (the latest stable version)

python -m pip install joeynmt

B. From source (for local development)

git clone https://github.com/joeynmt/joeynmt.git  # Clone this repository
cd joeynmt
python -m pip install -e .  # Install Joey NMT and it's requirements
python -m unittest  # Run the unit tests

Usage

For details, check out our documentation!

1. Torch Hub

import torch

model = torch.hub.load("joeynmt/joeynmt", "wmt14_ende")
translations = model.translate(["Hello world!"])
print(translations[0])  # "Hallo Welt!"

2. Command-line Interface

python -m joeynmt {train,test,translate} CONFIG_PATH [-o OUTPUT_PATH] [-a] [-s] [-t] [-d]

positional arguments:
  {train,test,translate}    Train a model or test or translate
  CONFIG_PATH               Path to YAML config file

options:
  -h, --help                show this help message and exit
  -o OUTPUT_PATH, --output-path OUTPUT_PATH
                            Path for saving translation output
  -a, --save-attention      Save attention visualizations
  -s, --save-scores         Save scores
  -t, --skip-test           Skip test after training
  -d, --use-ddp             Invoke DDP environment

For example:

• training

  python -m joeynmt train configs/transformer_small.yaml --use_ddp --skip-test
  

• testing

  python -m joeynmt test configs/transformer_small.yaml --output-path model_dir/hyp --save-scores --save-attention
  

• translation

  python -m joeynmt translate configs/transformer_small.yaml < input.txt > output.txt
  

Citation

If you use Joey NMT in a publication or thesis, please cite the following paper:

@inproceedings{kreutzer-etal-2019-joey,
    title = "Joey {NMT}: A Minimalist {NMT} Toolkit for Novices",
    author = "Kreutzer, Julia  and
      Bastings, Jasmijn  and
      Riezler, Stefan",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations",
    month = nov,
    year = "2019",
    address = "Hong Kong, China",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/D19-3019",
    doi = "10.18653/v1/D19-3019",
    pages = "109--114",
}