transquest 0.2.0

Transformer based translation quality estimation

TransQuest : Transformer based Translation Quality Estimation.

TransQuest provides state-of-the-art models for translation quality estimation.

Features

• Sentence-level translation quality estimation on both aspects: predicting post editing efforts and direct assessment.
• Perform significantly better than current state-of-the-art quality estimation methods like DeepQuest and OpenKiwi in all the languages experimented.
• Pre-trained quality estimation models for seven language-pairs.

Installation

You first need to install PyTorch. The recommended PyTorch version is 1.5. Please refer to PyTorch installation page regarding the specific install command for your platform.

When PyTorch has been installed, you can install TransQuest from source or from pip.

From Source

git clone https://github.com/TharinduDR/TransQuest.git
cd TransQuest
pip install -r requirements.txt

From pip

pip install transquest

Run the examples

Examples are included in the repository but are not shipped with the library.

1. WMT 2020 Sentence-level Direct Assessment QE Shared Task
2. WMT 2020 Sentence-level Post-Editing Effort QE Shared Task
3. WMT 2019 Sentence-level Post-Editing Effort QE Shared Task

Pre-trained models

Following pre-trained models are released. We will be keep releasing new models. Please keep in touch.

Language Pair	Objective	Algorithm	Model Link	Data	Pearson	MAE	RMSE
Romanian-English	Direct	TransQuest	model.zip	WMT 2020	0.8982	0.3121	0.4097
		SiameseTransQuest		WMT 2020	0.8501	0.3637	0.4932
		OpenKiwi		WMT 2020	0.6845	0.7596	1.0522
Estonian-English	Direct	TransQuest	model.zip	WMT 2020	0.7748	0.5904	0.7321
		SiameseTransQuest	model.zip	WMT 2020	0.6804	0.7047	0.9022
		OpenKiwi		WMT 2020	0.4770	0.9176	1.1382
Nepalese-English	Direct	TransQuest	model.zip	WMT 2020	0.7914	0.3975	0.5078
		SiameseTransQuest	model.zip	WMT 2020	0.6081	0.6531	0.7950
		OpenKiwi		WMT 2020	0.3860	0.7353	0.8713
Sinhala-English	Direct	TransQuest	model.zip	WMT 2020	0.6525	0.4510	0.5570
		SiameseTransQuest		WMT 2020
		OpenKiwi		WMT 2020	0.3737	0.7517	0.8978
Russian-English	Direct	TransQuest	model.zip	WMT 2020	0.7734	0.5076	0.6856
		SiameseTransQuest		WMT 2020
		OpenKiwi		WMT 2020	0.5479	0.8253	1.1930
English-German	Direct	TransQuest	model.zip	WMT 2020	0.4669	0.6474	0.7762
		SiameseTransQuest		WMT 2020
		OpenKiwi		WMT 2020	0.1455	0.6791	0.9670
	HTER	TransQuest		WMT 2020	0.4994	0.1486	0.1842
		SiameseTransQuest		WMT 2020
		OpenKiwi		WMT 2020	0.3916	0.1500	0.1896
	HTER	TransQuest		WMT 2019
		SiameseTransQuest		WMT 2019
		OpenKiwi		WMT 2019
English-Chinese	Direct	TransQuest	model.zip	WMT 2020	0.4779	0.9865	1.1338
		SiameseTransQuest	model.zip	WMT 2020	0.4067	1.0389	1.1973
		OpenKiwi		WMT 2020	0.1676	0.6559	0.8503
	HTER	TransQuest		WMT 2020
		SiameseTransQuest		WMT 2020
		OpenKiwi		WMT 2020

Once downloading them and unzipping it, they can be loaded easily

model = QuestModel("xlmroberta", "path, num_labels=1,
                               use_cuda=torch.cuda.is_available(), args=transformer_config)

model = SiameseTransQuestModel("path")

Citation

Please consider citing us if you use the library.

Coming soon!
Please keep in touch