Skip to main content

Chinese Word Segmenter developed by Nanjing University NLP Group

Project description

1 Installation and Usage

1.1 Installation

Installation through pip:

pip install njuseg

1.2 Usage

from njuseg import Segmenter
segmenter = Segmenter.load_model(model_pth,use_gpu=True)
sentences = ['美国联邦储备委员会 16 日发布的全国经济形势调查报告显示,去年 12 月初至今年 1 月上旬,美国经济继续温和扩张,但美国企业对经济前景的乐观程度有所下降。','美联储注意到了市场对全球经济放缓等风险因素的担心,但当前美国经济发生衰退的风险并未上升。']
segmented_sentences = segmenter.seg(sentences)

2 Performance

2.1 In domain:

with pretrained unigram + bigram embedding

96.63 96.52 98.14 96.13 95.82

3 Speed

On CPU: 20 k characters per second On single NVIDIA GTX 1080 GPU: 160 k characters per second

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for njuseg, version 2.0
Filename, size File type Python version Upload date Hashes
Filename, size njuseg-2.0-py2.py3-none-any.whl (12.2 kB) File type Wheel Python version py2.py3 Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page