Skip to main content

Python tool to extract sentences from po files and create language datasets for machine translation

Project description

PO2Dataset

PyPI version PyPI Supported Python Versions check

po2dataset is a python tool to extract sentences from po files and create language datasets for machine translation.

This command line tool is intended to create dataset packages suitable for Argos Train.

How to install

From pip

pip install po2dataset

Manual installation

Create a virtual environment using virtualenv

git clone https://github.com/itzune/po2dataset.git
virtualenv po2dataset
cd po2dataset
source ./bin/activate

Quick start guide

Create Argos Train suitable dataset

po2dataset path/to/yourfile.po --name "MyProject" --source_code en --target_code eu --ref "Some reference information of the project"

Where:

  • name: The name of the project
  • source_code: Source language code (ISO 639)
  • target_code: Target language code (ISO 639)
  • ref: Some reference information of the project

Optional arguments:

  • format: Extension name of the zip file (default argosdata)
  • license: License to add into the package (default CC0). Options are: CC0, CC-BY, CC-BY-SA

Usage Examples

Basic Dataset Creation

To create a dataset from a .po file for an English-Basque translation project, run:

po2dataset path/to/yourfile.po --name "MyProject" --source_code en --target_code eu --ref "Translation dataset for project X"

Specifying Format and License

For different file format and license, use:

po2dataset path/to/yourfile.po --name "MyProject" --source_code en --target_code eu --format "zip" --license "CC-BY"

These commands create language dataset packages, with zip file format and CC-BY licensing options.

Support

Should you experience any issues do not hesistate to post an issue or contribute in this project pulling requests.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

po2dataset-0.5.1.tar.gz (7.8 kB view hashes)

Uploaded Source

Built Distribution

po2dataset-0.5.1-py3-none-any.whl (6.9 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page