transquest 0.1.0

Transformer based translation quality estimation

TransQuest : Transformer based Translation Quality Estimation.

TransQuest provides state-of-the-art models for Quality Estimation.

Installation

you first need to install PyTorch. Please refer to PyTorch installation page regarding the specific install command for your platform.

When PyTorch has been installed, you can install from source by cloning the repository and running:

git clone https://github.com/TharinduDR/TransQuest.git
cd TransQuest
pip install -r requirements.txt

Run the examples

Examples are included in the repository but are not shipped with the library.

Romanian - English

python -m examples.ro_en.trans_quest

Algo Type	Transformer Type	Transformer Name	Pearson Correlation
TransQuest	XLM	xlm-mlm-enro-1024	0.739
TransQuest	XLM-R	xlm-roberta-large	0.894
TransQuest	BERT	bert-base-multilingual-cased	0.829
TransQuest	DistilBERT	distilbert-base-multilingual-cased	0.778

Russian - English

python -m examples.ru_en.trans_quest

Algo Type	Transformer Type	Transformer Name	Pearson Correlation
TransQuest	XLM-R	xlm-roberta-base	0.692
TransQuest	XLM-R	xlm-roberta-large	0.715
TransQuest	BERT	bert-base-multilingual-cased	0.642
TransQuest	DistilBERT	distilbert-base-multilingual-cased	0.644

Estonian - English

python -m examples.et_en.trans_quest

Algo Type	Transformer Type	Transformer Name	Pearson Correlation
TransQuest	XLM-R	xlm-roberta-base	0.672
TransQuest	XLM-R	xlm-roberta-large	0.741
TransQuest	BERT	bert-base-multilingual-cased	0.664

English - Chinese

python -m examples.en_zh.trans_quest

Algo Type	Transformer Type	Transformer Name	Pearson Correlation
TransQuest	XLM-R	xlm-roberta-base	0.493
TransQuest	XLM-R	xlm-roberta-large	0.501
TransQuest	BERT	bert-base-multilingual-cased	0.518

Nepalese - English

python -m examples.si_en.trans_quest

Algo Type	Transformer Type	Transformer Name	Pearson Correlation
TransQuest	XLM-R	xlm-roberta-base	0.699
TransQuest	XLM-R	xlm-roberta-large	0.761
TransQuest	BERT	bert-base-multilingual-cased	0.684

English - German

python -m examples.en_de.trans_quest

Algo Type	Transformer Type	Transformer Name	Pearson Correlation
TransQuest	XLM	xlm-mlm-ende-1024	0.326
TransQuest	BERT	bert-base-multilingual-cased	0.449

Sinhala - English

python -m examples.si_en.trans_quest

Algo Type	Transformer Type	Transformer Name	Pearson Correlation
TransQuest	XLM-R	xlm-roberta-base	0.380
TransQuest	XLM-R	xlm-roberta-large	0.589