한국어 용언 분석기 (Korean Lemmatizer)
한국어의 동사와 형용사의 활용형 (surfacial form) 을 분석합니다. 한국어 용언 분석기는 다음의 기능을 제공합니다.
- 입력된 단어를 어간 (stem) 과 어미 (eomi) 으로 분리
- 입력된 단어를 원형으로 복원
이 패키지의 구현 원리는 github.io 블로그에 정리하였습니다.
Usage
analyze, lemmatize, conjugate
analyze function returns morphemes of the given predicator word
from soylemma import Lemmatizer
lemmatizer = Lemmatizer()
lemmatizer.analyze('차가우니까')
The return value forms list of tuples because there can be more than one morpheme combination.
[(('차갑', 'Adjective'), ('우니까', 'Eomi'))]
lemmatize function returns lemma of the given predicator word.
lemmatizer.lemmatize('차가우니까')
[('차갑다', 'Adjective')]
If the input word is not predicator such as Noun, it return empty list.
lemmatizer.lemmatize('한국어') # []
conjugate function returns surfacial form. You should put stem and eomi as arguments. It returns all possible surfacial forms for the given stem and eomi.
lemmatizer.conjugate(stem='차갑', eomi='우니까')
lemmatizer.conjugate('예쁘', '었던')
['차가우니까', '차갑우니까']
['예뻤던', '예쁘었던']
update dictionaries and rules
For demonstration, we use dictioanry demo.
어여뻤어 cannot be analyzed because the adjective 어여쁘 does not enrolled in dictionary.
from soylemma import Lemmatizer
lemmatizer = Lemmatizer(dictionary_name='demo')
print(lemmatizer.analyze('어여뻤어')) # []
So, we add the word with tag using add_words function. Do it again. Then you can see the word 어여뻤어 is analyzed.
lemmatizer.add_words('어여쁘', 'Adjective')
lemmatizer.analyze('어여뻤어')
[(('어여쁘', 'Adjective'), ('었어', 'Eomi'))]
However, the word 파랬다 is still not able to be analyzed because the lemmatization rule for surfacial form 랬 does not exist.
lemmatizer.analyze('파랬다') # []
So, in this time, we update additional lemmatization rules using add_lemma_rules function.
supplements = {
'랬': {('랗', '았')}
}
lemmatizer.add_lemma_rules(supplements)
After that, we can see the word 파랬다 is analyzed, and also conjugation of 파랗 + 았다 is available.
lemmatizer.analyze('파랬다')
lemmatizer.conjugate('파랗', '았다')
[(('파랗', 'Adjective'), ('았다', 'Eomi'))]
['파랬다', '파랗았다']
debug on
If you wonder which subwords came up as candidates of (stem, eomi), use debug.
lemmatizer.analyze('파랬다', debug=True)
[DEBUG] word: 파랬다 = 파랗 + 았다, conjugation: 랬 = 랗 + 았
[(('파랗', 'Adjective'), ('았다', 'Eomi'))]
lemmatization rule extractor
You can extract lemmatization rule using extract_rule function.
from soylemma import extract_rule
eojeol = '로드무비였다'
lw = '로드무비이'
lt = 'Adjective'
rw = '었다'
rt = 'Eomi'
extract_rule(eojeol, lw, lt, rw, rt)
('였다', ('이', '었다'))
Metadata
Release files for soylemma 0.2.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| soylemma-0.2.0.tar.gz | 125.2 kB | Without provenance | Details for soylemma-0.2.0.tar.gz |
Built distributions (wheels)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| soylemma-0.2.0-py3.7.egg | Legacy Egg format | - | - | Details for soylemma-0.2.0-py3.7.egg |
| soylemma-0.2.0-py3-none-any.whl | Python 3 | none | any | Details for soylemma-0.2.0-py3-none-any.whl |
Total release size: 384.2 kB
Release files / soylemma-0.2.0.tar.gz
| Download URL | soylemma-0.2.0.tar.gz |
|---|---|
| Size | 125.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
fa43afd27be7059fe4716774c928454f091e3a3b686a1f97c12e3d825960b6ec
|
|
BLAKE2b-256 checksum How to use checksums |
bf9a6fd9d5f475191aa834f8c75c5a94759fab5ebfc240e22262da43f006cbff
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.13.0 pkginfo/1.4.2 requests/2.21.0 setuptools/40.6.3 requests-toolbelt/0.9.1 tqdm/4.28.1 CPython/3.7.1
|
Release files / soylemma-0.2.0-py3.7.egg
| Download URL | soylemma-0.2.0-py3.7.egg |
|---|---|
| Size | 135.0 kB |
| Tags | Egg |
|
SHA-256 checksum How to use checksums |
8b2d4734c9b7669ae95810ff561acfd1ed0a86b26f1307cb1e777412196ded6c
|
|
BLAKE2b-256 checksum How to use checksums |
124fc2b97e5407d843f1aac8f7250fb5253ae788889972ff647089015d429b55
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.13.0 pkginfo/1.4.2 requests/2.21.0 setuptools/40.6.3 requests-toolbelt/0.9.1 tqdm/4.28.1 CPython/3.7.1
|
Release files / soylemma-0.2.0-py3-none-any.whl
| Download URL | soylemma-0.2.0-py3-none-any.whl |
|---|---|
| Size | 124.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
6767c3e35a45a475bf7635029eaa20ac5f492574b73952d1b6bfb78ed23a3a69
|
|
BLAKE2b-256 checksum How to use checksums |
03f5d612c52b363a1e04b2df6dbde86bedd84035ba3d07651c586dcfdac50b53
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.13.0 pkginfo/1.4.2 requests/2.21.0 setuptools/40.6.3 requests-toolbelt/0.9.1 tqdm/4.28.1 CPython/3.7.1
|
Security for soylemma 0.2.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.