Data Augmentation for Japanese Text
Project description
AugLy-jp
Data Augmentation for Japanese Text on AugLy
Augmenter
base_text = "あらゆる現実をすべて自分のほうへねじ曲げたのだ"
Augmenter | Augmented | Description |
---|---|---|
SynonymAugmenter | あらゆる現実をすべて自身のほうへねじ曲げたのだ | Substitute similar word according to Sudachi synonym |
WordEmbsAugmenter | あらゆる現実をすべて関心のほうへねじ曲げたのだ | Leverage word2vec, GloVe or fasttext embeddings to apply augmentation |
FillMaskAugmenter | つまり現実を、未来な未来まで変えたいんだ | Using masked language model to generate text |
BackTranslationAugmenter | そして、ほかの人たちをそれぞれの道に安置しておられた | Leverage two translation models for augmentation |
Prerequisites
Software | Install Command |
---|---|
Python 3.8.11 | pyenv install 3.8.11 |
Poetry 1.1.* | curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python |
Get Started
Installation
pip install augly-jp
Or clone this repository:
git clone https://github.com/chck/AugLy-jp.git
poetry install
Test with reformat
poetry run task test
Reformat
poetry run task fmt
Lint
poetry run task lint
Inspired
- https://github.com/facebookresearch/AugLy
- https://github.com/makcedward/nlpaug
- https://github.com/QData/TextAttack
License
This software includes the work that is distributed in the Apache License 2.0 [1].
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
augly_jp-2021.9.26.tar.gz
(10.7 kB
view hashes)
Built Distribution
Close
Hashes for augly_jp-2021.9.26-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c90ea9451a2944c84ad9c8789275fd7058876bb2fbb9f1ccd252368188c1ff5f |
|
MD5 | 44909ed19b4ddb92a17aef68f87f2e47 |
|
BLAKE2b-256 | 94f2ad9c6c9ba45cf080dbeaf0f220c068a2db797ba7bbabb9f45ab4d9f74af8 |