Skip to main content

Chinese Word Segmenter developed by Nanjing University NLP Group

Project description

1 Installation and Usage

1.1 Installation

Installation through pip:

pip install njuseg

1.2 Usage

from njuseg import Segmenter
segmenter = Segmenter.load_model(model_pth,use_gpu=True)
sentences = ['美国联邦储备委员会 16 日发布的全国经济形势调查报告显示,去年 12 月初至今年 1 月上旬,美国经济继续温和扩张,但美国企业对经济前景的乐观程度有所下降。','美联储注意到了市场对全球经济放缓等风险因素的担心,但当前美国经济发生衰退的风险并未上升。']
segmented_sentences = segmenter.seg(sentences)

2 Performance

2.1 In domain:

with pretrained unigram + bigram embedding

PKU MSR CTB5 CTB6 NLPCC
96.63 96.52 98.14 96.13 95.82

3 Speed

On CPU: 20 k characters per second On single NVIDIA GTX 1080 GPU: 160 k characters per second

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
njuseg-2.0-py2.py3-none-any.whl (12.2 kB) Copy SHA256 hash SHA256 Wheel py2.py3

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page