No project description provided
Project description
detransliterator
Table of Contents
Installation
pip install detransliterator
Usage
as a software library
from detransliterator import Detransliterator
detransliterator = Detransliterator('latin2nqo_001.35')
latin = "musa dunbuya"
nqo = detransliterator.detransliterate(latin, beam_size=5)
assert nqo == "ߡߎߛߊ߫ ߘߎ߲ߓߎߦߊ"
as a console tool
# detransliteration tool
python -m detransliterator.tool --help
# csv/tsv extraction tool
python -m detransliterator.csv_tool --help
example: detransliterate a stream
echo "musa dunbuya" | python -m detransliterator.tool --model-name latin2nqo_001.35
example: extract a column from a csv file
cat test_csv_no_header.csv \
| python -m detransliterator.csv_tool \
extract-column --column-ix 1 \
> tmp_col1.txt
example: extract a column from a tsv file
cat test_tsv_with_header.tsv \
| python -m detransliterator.csv_tool \
extract-column --column-ix 1 --skip-lines 1 --csv-formatting-params delimiter tab \
> tmp_col2.txt
example: detransliterate a column from a csv file
cat test_tsv_no_header.tsv \
| python -m detransliterator.csv_tool extract-column --column-ix 1 \
--csv-formatting-params delimiter tab \
| python -m detransliterator.tool --model-name latin2nqo_001.35 \
> tmp_detransliterated_1.nqo
example: detransliterate a column from a tsv file
cat test_csv_with_header.csv \
| python -m detransliterator.csv_tool extract-column --column-ix 1 --skip-lines 1 \
| python -m detransliterator.tool --model-name latin2nqo_001.35 \
> tmp_detransliterated_2.nqo
example: use a particular GPU
CUDA_VISIBLE_DEVICES="1" echo "musa dunbuya" | python -m detransliterator.tool
Detransliteration Models
Model | Source Script | Target Script | #Parameters | Validation BLEU | Test BLEU |
---|---|---|---|---|---|
latin2nqo_001.35 | latin | nqo | 2 520 576 | 75.56 | 74.14 |
latin2nqo_001.38 | latin | nqo | 3 909 120 | 78.51 | 77.06 |
License
detransliterator
is distributed under the terms of the MIT license.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
detransliterator-0.0.2.tar.gz
(74.3 MB
view hashes)
Built Distribution
Close
Hashes for detransliterator-0.0.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a5bb046002502481a9c4b2dd39a61527adc14def7592aa365b0a9d2ac2d96d23 |
|
MD5 | 9f6e22eb5aefd9bb008df1cd83aa9a74 |
|
BLAKE2b-256 | 7c320bf1b166a3fd87f720541856a23d327b494ab68c7f8ebef4fae411cce553 |