BackTranslation 0.5.0

Back translation for Natural Language Processing (NLP) using Google Translate

BackTranslation

BackTranslation is a python library that implements back translation and direct translation among languages. It utilizes the googletrans library and Baidu Translation API to translate text.

Since there is an error in current verison of googletrans, you have to create only one instance to do back-translation for your work. Otherwise, it is easy to cause a bug from multi-requests. We will keep implementing this library with other translator libraries soon.

If you face any bug, you can open a issue in Github.

Installation

You can install it from PyPI:

$ pip install BackTranslation

Usage

Direct translation with googletrans

Translate text from language A directly to language B without translating it back.

Parameters:

• url: option. provide a list of services urls for translation if need. Default url is translate.google.com.
• proxies: Optional. Proxies configuration. Dictionary mapping protocol or protocol and host to the URL of the proxy. i.e. proxies = {'http': '127.0.0.1:1234', 'http://host.name': '127.0.0.1:4012'}
• text: required. Original text that need to translate.
• src: option. Source language code of original text. If this parameter is None, the method will detect the language of text automatically. (Default: None)
• dst: required. Destination language code.
• sleeping: option. Kept for API consistency. Direct translation only sends the A-to-B request. (Default: 0)

Return parameter: object Translated.

Attributes:

• source_text: original sentence.
• src: the language of original sentence.
• dst: the target language.
• tmp: the same as dst for direct translation.
• tran_text: translated result in dst language.
• result_text: translated result in dst language.
• mode: direct.

from BackTranslation import BackTranslation
trans = BackTranslation()
result = trans.direct_translate('hello', src='en', dst='zh-cn')
print(result.result_text)
# '你好'

Backtranslation with googletrans

Translate the original text to other language and translate back to augment the diversity of data in NLP research.

Parameters:

• url: option. provide a list of services urls for translation if need. Default url is translate.google.com.
• proxies: Optional. Proxies configuration. Dictionary mapping protocol or protocol and host to the URL of the proxy. i.e. proxies = {'http': '127.0.0.1:1234', 'http://host.name': '127.0.0.1:4012'}
• text: required. Original text that need to do back translation.
• src: option. Source language code of original text. If this parameter is None, the method will detect the language of text automatically. (Default: None)
• tmp: option. Middle language code. If this parameter is None, the method will pick one of two languages which is different from src.
• sleeping: option. It is a timer to limit the speed of back-translation to avoid Google rate limits (HTTP 429). Increase this value if you encounter errors after many translations. (Default: 0)

Return parameter: object Translated.

Attributes:

• source_text: original sentence.
• src: the language of original sentence
• tmp: the target language as middle man
• dst: the same as tmp for back-translation
• tran_text: intermediate result
• result_text: back-translated result
• mode: back_translation

from BackTranslation import BackTranslation
trans = BackTranslation()
result = trans.translate('hello', src='en', tmp='zh-cn')
print(result.result_text)
# 'Hello there'

Complete example with auto language detection:

from BackTranslation import BackTranslation
trans = BackTranslation()
result = trans.translate('Anh ấy đã chữa khỏi cảm cúm bằng aspirin.')
print(result.src)         # 'vi'
print(result.tmp)         # 'en'
print(result.tran_text)   # intermediate translation
print(result.result_text) # back-translated result

If Google blocks your IP, you can provide alternative service URLs or a proxy:

from BackTranslation import BackTranslation
trans = BackTranslation(url=[
      'translate.google.com',
      'translate.google.co.kr',
    ], proxies={'http': '127.0.0.1:1234', 'http://host.name': '127.0.0.1:4012'})
result = trans.translate('hello', src='en', tmp='zh-cn')
print(result.result_text)

Note: You just need to create one instance of BackTranslation in order to avoid the issue in current version of googletrans.

Search the language code

You may find out your language code with full language name by using this method.

Parameters:

• language: required. A language name in english.

from BackTranslation import BackTranslation
trans = BackTranslation()
trans.searchLanguage('Chinese')
# {'chinese (simplified)': 'zh-cn', 'chinese (traditional)': 'zh-tw', 'chinese (cantonese)': 'yue'}
trans.searchLanguage('Cantonese')
# {'chinese (cantonese)': 'yue'}

Backtranslation_Baidu with Baidu Translation API

To use this stable translation, you are required to register in Baidu Translation API for getting your own appID. It supports 2 million chacters per day for free. Note: Currently, they only support Chinese phone number to register the accout.

• sleeping: option. Baidu standard API allows only 1 request per second (QPS limit). Set sleeping=1 (default) to stay within the limit. Increase if you encounter errors. (Default: 1)

Direct translation:

from BackTranslation import BackTranslation_Baidu
trans = BackTranslation_Baidu(appid='YOUR APPID', secretKey='YOUR SECRETKEY')
result = trans.direct_translate('hello', src='en', dst='zh')
print(result.result_text) # direct translated result
trans.closeHTTP()

Back-translation:

from BackTranslation import BackTranslation_Baidu
trans = BackTranslation_Baidu(appid='YOUR APPID', secretKey='YOUR SECRETKEY')
result = trans.translate('hello', src='auto', tmp='zh')
print(result.tran_text)   # intermediate translation
print(result.result_text) # back-translated result
trans.closeHTTP()

Seach language code

Since Baidu provides the different language code, it will be updated soon.

Version Information

Version 0.5.0: add direct translation APIs for Google and Baidu translators, keep existing back-translation behavior compatible, extend Translated with dst and mode, add Cantonese (yue) language support, and update tests and documentation.

Version 0.4.0: fix bugs (#1 #2 #3 #4 #6), add automated tests and CI/CD pipeline.

Version 0.3.1: fix some bugs for Baidu translator.

Version 0.2.2: fix the services url for Google Translator.

Version 0.2.1: fix the small bug. From this version, the library googletrans version is 4.0.0rc1.

Version 0.2.0: support back-translation with Baidu API, and fix bugs

Version 0.1.0: support back-translation with googletrans library

Contribution

Welcome to contribute BackTranslation library!

reference

• googletrans
• Baidu Translation API