ekorpkit 0.1.30

eKorpkit provides a flexible interface for corpus management and analysis pipelines such as extraction, transformation, tokenization, training, and visualization.

ekorpkit[iːkɔːkɪt]: (e)nglish (K)orean C(orp)us Tool(kit)

eKorpkit provides a flexible interface for corpus management and analysis pipelines such as extraction, transformation, tokenization, training, and visualization.

• Powerful config composition backed by Hydra - Easily swap out corpora, datasets, models, preprocessors, visualizers and many more configurations without touching the code.

Tutorials

Tutorials for ekorpkit package can be found at https://entelecheia.github.io/ekorpkit-config/

Installation

Install the latest version of ekorpit:

pip install ekorpkit

To install all extra dependencies,

pip install ekorpkit[all]

The eKorpkit Corpus

The eKorpkit Corpus is a large, diverse, bilingual (ko/en) language modelling dataset.

Citation

@software{lee_2022_6497226,
  author       = {Young Joon Lee},
  title        = {eKorpkit: English Korean Corpus Toolkit},
  month        = apr,
  year         = 2022,
  publisher    = {Zenodo},
  doi          = {10.5281/zenodo.6497226},
  url          = {https://doi.org/10.5281/zenodo.6497226}
}

@software{lee_2022_ekorpkit,
  author       = {Young Joon Lee},
  title        = {eKorpkit: English Korean Corpus Toolkit},
  month        = apr,
  year         = 2022,
  publisher    = {GitHub},
  url          = {https://github.com/entelecheia/ekorpkit}
}

License

• eKorpkit is licensed under the Creative Commons License(CCL) 4.0 CC-BY. This license covers the eKorpkit package and all of its components.
• Each corpus adheres to its own license policy. Please check the license of the corpus before using it!