A unified language analyzer for Japanese
Project description
KWJA: Kyoto-Waseda Japanese Analyzer
KWJA is a Japanese language analyzer based on pre-trained language models. KWJA performs many language analysis tasks, including:
- Typo correction
- Tokenization
- Morphological analysis
- Named entity recognition
- Dependency parsing
- PAS analysis
- Coreference resolution
- Discourse relation analysis
- etc.
Requirements
- Python: 3.9+
- Dependencies: See pyproject.toml.
Getting Started
Install KWJA with pip:
$ pip install kwja
Perform language analysis with the kwja
command (the result is in the KNP format):
# Analyze a text
$ kwja --text "KWJAは日本語の統合解析ツールです。汎用言語モデルを利用し、様々な言語解析を統一的な方法で解いています。"
# Analyze a text file
$ kwja --file path/to/file.txt
Usage from Python
Make sure you have kwja
command in your path:
$ which kwja
/path/to/kwja
Install rhoknp:
$ pip install rhoknp
Perform language analysis with the kwja
instance:
from rhoknp import KWJA
kwja = KWJA()
analyzed_document = kwja.apply(
"KWJAは日本語の統合解析ツールです。汎用言語モデルを利用し、様々な言語解析を統一的な方法で解いています。"
)
Citation
@InProceedings{植田2022,
author = {植田 暢大 and 大村 和正 and 児玉 貴志 and 清丸 寛一 and 村脇 有吾 and 河原 大輔 and 黒橋 禎夫},
title = {KWJA:汎用言語モデルに基づく日本語解析器},
booktitle = {第253回自然言語処理研究会},
year = {2022},
address = {京都},
}
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
kwja-1.0.2.tar.gz
(18.1 MB
view hashes)
Built Distribution
kwja-1.0.2-py3-none-any.whl
(18.2 MB
view hashes)