Python tool to extract sentences from po files and create language datasets for machine translation
Project description
PO2Dataset
po2dataset is a python tool to extract sentences from po files and create language datasets for machine translation.
This command line tool is intended to create dataset packages suitable for Argos Train.
How to install
From pip
pip install po2dataset
Manual installation
Create a virtual environment using virtualenv
git clone https://github.com/itzune/po2dataset.git
virtualenv po2dataset
cd po2dataset
source ./bin/activate
Quick start guide
Create Argos Train suitable dataset
po2dataset path/to/yourfile.po --name "MyProject" --source_code en --target_code eu --ref "Some reference information of the project"
Where:
name
: The name of the projectsource_code
: Source language code (ISO 639)target_code
: Target language code (ISO 639)ref
: Some reference information of the project
Optional arguments:
format
: Extension name of the zip file (default argosdata)license
: License to add into the package (default CC0). Options are: CC0, CC-BY, CC-BY-SA
Usage Examples
Basic Dataset Creation
To create a dataset from a .po file for an English-Basque translation project, run:
po2dataset path/to/yourfile.po --name "MyProject" --source_code en --target_code eu --ref "Translation dataset for project X"
Specifying Format and License
For different file format and license, use:
po2dataset path/to/yourfile.po --name "MyProject" --source_code en --target_code eu --format "zip" --license "CC-BY"
These commands create language dataset packages, with zip file format and CC-BY licensing options.
Support
Should you experience any issues do not hesistate to post an issue or contribute in this project pulling requests.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for po2dataset-0.5.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1a1c9bd7a7570305e5a084f62b226a7470fd7d5dab80055e1a710783ea754545 |
|
MD5 | 68f8f3db9c2a3e673a0785116f071c67 |
|
BLAKE2b-256 | d224ddd6a21e5dda4c81942658d7a28e4bea208340912df153fad69ce06b99f9 |