eKoNLPy 2.2.0

A Korean natural language processing toolkit for economic analysis

eKoNLPy: Korean NLP Python Library for Economic Analysis

eKoNLPy is a Korean Natural Language Processing (NLP) Python library specifically designed for economic analysis. It extends the functionality of the Mecab tagger from KoNLPy to improve the handling of economic terms, financial institutions, and company names by classifying them as single nouns. Additionally, it incorporates sentiment analysis features to determine the tone of monetary policy statements, such as hawkish or dovish.

Note

From version 2.0.0, eKoNLPy integrates the extended tagger with the original tagger. If you want to use the original tagger, set use_original_tagger=True when creating an instance of the Mecab class. Additionally, the Mecab class can be directly imported from the ekonlpy module. The default input text parameter of Mecab.pos() has been changed from phrase to text to be consistent with the original tagger.

Note

eKoNLPy is built on the fugashi and mecab-ko-dic libraries. For more information on using the Mecab tagger, refer to the fugashi documentation. Since eKoNLPy no longer relies on the KoNLPy library, Java is not required for its use. This makes eKoNLPy compatible with Windows, Linux, and macOS without the need for Java installation. You can also use eKoNLPy on Google Colab.

If you wish to tokenize general Korean text with eKoNLPy, you do not need to install the KoNLPy library. Instead, use the same ekonlpy.Mecab class with the use_original_tagger=True option.

However, if you plan to use the Korean Sentiment Analyzer (KSA), which employs the Kkma morpheme analyzer, you will need to install the KoNLPy library.

Installation

To install eKoNLPy, run the following command:

pip install ekonlpy

Usage

Part of Speech Tagging

To use the part-of-speech tagging feature, input Mecab.pos(text) just like KoNLPy. First, the input is processed using KoNLPy's Mecab morpheme analyzer. Then, if a combination of consecutive tokens matches a term in the user dictionary, the phrase is separated into compound nouns.

from ekonlpy import Mecab

mecab = Mecab()
mecab.pos('금통위는 따라서 물가안정과 병행, 경기상황에 유의하는 금리정책을 펼쳐나가기로 했다고 밝혔다.')

[('금', 'MAJ'), ('통', 'MAG'), ('위', 'NNG'), ('는', 'JX'), ('따라서', 'MAJ'), ('물가', 'NNG'), ('안정', 'NNG'), ('과', 'JC'), ('병행', 'NNG'), (',', 'SC'), ('경기', 'NNG'), ('상황', 'NNG'), ('에', 'JKB'), ('유의', 'NNG'), ('하', 'XSV'), ('는', 'ETM'), ('금리', 'NNG'), ('정책', 'NNG'), ('을', 'JKO'), ('펼쳐', 'VV+EC'), ('나가', 'VX'), ('기', 'ETN'), ('로', 'JKB'), ('했', 'VV+EP'), ('다고', 'EC'), ('밝혔', 'VV+EP'), ('다', 'EF'), ('.', 'SF')]

You can also use the Command Line Interface (CLI) to perform part-of-speech tagging:

ekonlpy --input "안녕하세요"

[('안녕', 'NNG'), ('하', 'XSV'), ('세요', 'EP')]

Original Mecab POS Tagging (fugashi)

from ekonlpy import Mecab

mecab = Mecab(use_original_tagger=True) # set use_original_tagger=True
mecab.pos('금통위는 따라서 물가안정과 병행, 경기상황에 유의하는 금리정책을 펼쳐나가기로 했다고 밝혔다.')

[('금', 'MAJ'), ('통', 'MAG'), ('위', 'NNG'), ('는', 'JX'), ('따라서', 'MAJ'), ('물가', 'NNG'), ('안정', 'NNG'), ('과', 'JC'), ('병행', 'NNG'), (',', 'SC'), ('경기', 'NNG'), ('상황', 'NNG'), ('에', 'JKB'), ('유의', 'NNG'), ('하', 'XSV'), ('는', 'ETM'), ('금리', 'NNG'), ('정책', 'NNG'), ('을', 'JKO'), ('펼쳐', 'VV+EC'), ('나가', 'VX'), ('기', 'ETN'), ('로', 'JKB'), ('했', 'VV+EP'), ('다고', 'EC'), ('밝혔', 'VV+EP'), ('다', 'EF'), ('.', 'SF')]

Lemmatization and Synonyms

To enhance the accuracy of sentiment analysis, eKoNLPy offers lemmatization and synonym handling features.

Adding Words to Dictionary

You can add words to the dictionary in the ekonlpy.tag module's Mecab class, either as a string or a list of strings, using the add_dictionary method.

from ekonlpy.tag import Mecab

mecab = Mecab()
mecab.add_dictionary('금통위', 'NNG')

Sentiment Analysis

Korean Monetary Policy Dictionary (MPKO)

To perform sentiment analysis using the Korean Monetary Policy dictionary, create an instance of the MPKO class in ekonlpy.sentiment:

from ekonlpy.sentiment import MPKO

mpko = MPKO(kind=1)
tokens = mpko.tokenize(text)
score = mpko.get_score(tokens)

The kind parameter in the MPKO class is used to select a lexicon file:

• 0: A lexicon file generated using a Naive-Bayes classifier with 5-gram tokens as features and changes in call rates as positive/negative labels.
• 1: A lexicon file generated using polarity induction and seed propagation methods with 5-gram tokens.

Korean Monetary Policy Classifier (MPCK)

To use a classifier for monetary policy sentiment analysis, use the MPCK class from ekonlpy.sentiment:

from ekonlpy.sentiment import MPCK

mpck = MPCK()
tokens = mpck.tokenize(text)
ngrams = mpck.ngramize(tokens)
score = mpck.classify(tokens + ngrams, intensity_cutoff=1.5)

You can set the intensity_cutoff parameter to adjust the intensity threshold for classifying low-confidence sentences as neutral (default: 1.3).

Korean Sentiment Analyzer (KSA)

For general Korean sentiment analysis, use the KSA class. The morpheme analyzer used in this class is Kkma, developed by Seoul National University's IDS Lab. The sentiment dictionary is also from the same lab (reference: http://kkma.snu.ac.kr/).

from ekonlpy.sentiment import KSA

ksa = KSA()
tokens = ksa.tokenize(text)
score = ksa.get_score(tokens)

Harvard IV-4 Dictionary

For general English sentiment analysis, use the Harvard IV-4 dictionary:

from ekonlpy.sentiment import HIV4

hiv = HIV4()
tokens = hiv.tokenize(text)
score = hiv.get_score(tokens)

Loughran and McDonald Dictionary

For sentiment analysis in the financial domain, use the Loughran and McDonald dictionary:

from ekonlpy.sentiment import LM

lm = LM()
tokens = lm.tokenize(text)
score = lm.get_score(tokens)

Changelog

See the CHANGELOG for more information.

Contributing

Contributions are welcome! Please see the contributing guidelines for more information.

License

eKoNLPy is an open-source software library for Korean Natural Language Processing (NLP), specifically designed for economic analysis. The library is released under the MIT License, allowing developers and researchers to use, modify, and distribute the software freely.

Citation

If you use eKoNLPy in your work or research, please cite the following sources:

• Lee, Young Joon, eKoNLPy: A Korean NLP Python Library for Economic Analysis, 2018. Available at: https://github.com/entelecheia/eKoNLPy.
• Lee, Young Joon, Soohyon Kim, and Ki Young Park. "Deciphering Monetary Policy Board Minutes with Text Mining: The Case of South Korea." Korean Economic Review 35 (2019): 471-511.

You can also use the following BibTeX entry for citation:

@misc{lee2018ekonlpy,
    author= {Lee, Young Joon},
    year  = {2018},
    title = {{eKoNLPy: A Korean NLP Python Library for Economic Analysis}},
    note  = {\url{https://github.com/entelecheia/eKoNLPy}}
}

By citing eKoNLPy in your work, you acknowledge the efforts and contributions of its creators and help promote further development and research in Korean NLP for economic analysis.