Textmentations
Textmentations is a Python library for augmenting Korean text. Inspired by albumentations. Textmentations uses the albumentations as a dependency.
Installation
pip install textmentations
A simple example
Textmentations provides text augmentation techniques implemented using the TextTransform, which inherits from the albumentations BasicTransform.
This allows textmentations to reuse the existing functionalities of albumentations.
import textmentations as T
text = "어제 식당에 갔다. 목이 너무 말랐다. 먼저 물 한 잔을 마셨다. 그리고 탕수육을 맛있게 먹었다."
rd = T.RandomDeletion(deletion_prob=0.1, min_words_per_sentence=0.8)
ri = T.RandomInsertion(insertion_prob=0.2, n_times=1)
rs = T.RandomSwap(alpha=1)
sr = T.SynonymReplacement(replacement_prob=0.2)
eda = T.Compose([rd, ri, rs, sr])
print(rd(text=text)["text"])
# 식당에 갔다. 목이 너무 말랐다. 먼저 물 잔을 마셨다. 그리고 탕수육을 맛있게 먹었다.
print(ri(text=text)["text"])
# 어제 최근 식당에 갔다. 목이 너무 말랐다. 먼저 물 한 잔을 마셨다 음료수. 그리고 탕수육을 맛있게 먹었다.
print(rs(text=text)["text"])
# 어제 갔다 식당에. 목이 너무 말랐다. 물 먼저 한 잔을 마셨다. 그리고 탕수육을 맛있게 먹었다..
print(sr(text=text)["text"])
# 과거 식당에 갔다. 목이 너무 말랐다. 먼저 소주 한 잔을 마셨다. 그리고 탕수육을 맛있게 먹었다.
print(eda(text=text)["text"])
# 식당에 어제 과거 갔다. 너무 말랐다. 먼저 상수 한 잔을 마셨다 맹물. 그리고 맛있게 먹었다.
List of augmentations
- AEDA
- BackTranslation
- ContextualInsertion
- ContextualReplacement
- IterativeMaskFilling
- RandomDeletion
- RandomDeletionSentence
- RandomInsertion
- RandomSwap
- RandomSwapSentence
- SynonymReplacement
References
- AEDA: An Easier Data Augmentation Technique for Text Classification
- Conditional BERT Contextual Augmentation
- Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relations
- EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks
- Iterative Mask Filling: An Effective Text Augmentation Method Using Masked Language Modeling
- Korean Stopwords
- Korean WordNet
- albumentations
- kykim/albert-kor-base
1 maintainer
Metadata
1 maintainer
Release files for textmentations 1.4.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| textmentations-1.4.0.tar.gz | 49.8 MB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| textmentations-1.4.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 99.7 MB
Release files / textmentations-1.4.0.tar.gz
| Download URL | textmentations-1.4.0.tar.gz |
|---|---|
| Size | 49.8 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
77b1816077c08cc2956f698d8a9825804f09b369a89e4ab50ca36e27839930b1
|
|
BLAKE2b-256 checksum How to use checksums |
2b94eb13efb0b226fab8d120c69ed4560b8d39100f30e7e49283e339d46d3a1d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.1.1 CPython/3.10.14
|
Release files / textmentations-1.4.0-py3-none-any.whl
| Download URL | textmentations-1.4.0-py3-none-any.whl |
|---|---|
| Size | 49.8 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
0cde4484974ac184c88cc96cb17f304625c7357ff6bbb4d73c3cf341b156e11d
|
|
BLAKE2b-256 checksum How to use checksums |
11f31953d24f57ebaf7997b65e81df8fbca4bfa6540bae88b8a6680d595f2749
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.1.1 CPython/3.10.14
|
