Skip to main content

No project description provided

Project description

NLP JP Gears

PyPI version

Overview

日本語の自然言語処理で頻出の前処理をまとめたものです。pipelineをしいて、複数の処理をまとめることができます。

API

  • pipelineの作成: composer.Composer
  • 全角英数字記号を半角に変換: zenhan.ZenToHanConverter
  • 半角英数字記号を全角に変換: zenhan.HanToZenConverter
  • 括弧とその間のテキストを削除: remover.TextBtwBracketsRemover

Requirements

Python 3.6+

Installation

pip install nlp-jp-gears

Example

from nlp_jp_gears import Composer
from nlp_jp_gears import (
    ZenToHanConverter,
    TextBtwBracketsRemover
)

txt_btw_brackets_remover = TextBtwBracketsRemover()
zenhan_converter = ZenToHanConverter()

composer = Composer(txt_btw_brackets_remover, zenhan_converter)
text = "Python(パイソン)で自然言語処理?"
out = composer(text)
print(out)

Then, input text is preprocessed.

Pythonで自然言語処理?

And you can check what is removed and converted, as follows,

print(txt_btw_brackets_remover.removes)
print(zenhan_converter.converts)
<{[(「『([〈《〔{«‹

!"#$%&'()*+,-./:;<=>?@[\]^_`{|}~0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for nlp-jp-gears, version 0.1.1
Filename, size File type Python version Upload date Hashes
Filename, size nlp_jp_gears-0.1.1-py3-none-any.whl (5.5 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size nlp-jp-gears-0.1.1.tar.gz (4.3 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page