ckipnlp 0.5.1

CKIP CoreNLP Wrappers

Introduction

Git

https://github.com/emfomy/ckipnlp

PyPI

https://pypi.org/project/ckipnlp

Author

• Mu Yang <[e-mail]>

Documentation

http://ckipnlp.readthedocs.io/

Requirements

• Python 3.5+

• Cython 0.29+

CkipWs (Optional)

• CKIP Word Segmentation Linux version (20190524+)

CkipParser (Optional)

• CKIP Parser Linux version (20190506+)

• Boost C++ Libraries 1.54.0

Installation

Denote <ckipws-linux-root> as the root path of CKIPWS Linux Version, and <ckipparser-linux-root> as the root path of CKIP-Parser Linux Version.

Step 1: Setup CKIPWS & CKIP-Parser environment

Add below command to ~/.bashrc:

export LD_LIBRARY_PATH=<ckipws-linux-root>/lib:<ckipparser-linux-root>/lib:$LD_LIBRARY_PATH

Step 2: Install Using Pip

pip install ckipnlp \
   --install-option='--ws' \
   --install-option='--ws-dir=<ckipws-linux-root>' \
   --install-option='--parser' \
   --install-option='--parser-dir=<ckipparser-linux-root>'

Ignore ws/parser options if one doesn’t have CKIPWS/CKIP-Parser.

Installation Options

Option	Detail	Default Value
--[no-]ws	Enable/disable CKIPWS.	False
--[no-]parser	Enable/disable CKIP-Parser.	False
--ws-dir=<ws-dir>	CKIPWS root directory.
--ws-lib-dir=<ws-lib-dir>	CKIPWS libraries directory	<ws-dir>/lib
--ws-share-dir=<ws-share-dir>	CKIPWS share directory	<ws-dir>
--parser-dir=<parser-dir>	CKIP-Parser root directory.
--parser-lib-dir=<parser-lib-dir>	CKIP-Parser libraries directory	<parser-dir>/lib
--parser-share-dir=<parser-share-dir>	CKIP-Parser share directory	<parser-dir>
--data2-dir=<data2-dir>	“Data2” directory	<ws-share-dir>/Data2
--rule-dir=<rule-dir>	“Rule” directory	<parser-share-dir>/Rule
--rdb-dir=<rdb-dir>	“RDB” directory	<parser-share-dir>/RDB

Usage

See http://ckipnlp.readthedocs.io/ for API details.

CKIPWS

import ckipnlp.ws
print(ckipnlp.__name__, ckipnlp.__version__)

ws = ckipnlp.ws.CkipWs(logger=False)
print(ws('中文字喔'))
for l in ws.apply_list(['中文字喔', '啊哈哈哈']): print(l)

ws.apply_file(ifile='sample/sample.txt', ofile='output/sample.tag', uwfile='output/sample.uw')
with open('output/sample.tag') as fin:
    print(fin.read())
with open('output/sample.uw') as fin:
    print(fin.read())

CKIP-Parser

import ckipnlp.parser
print(ckipnlp.__name__, ckipnlp.__version__)

ps = ckipnlp.parser.CkipParser(logger=False)
print(ps('中文字喔'))
for l in ps.apply_list(['中文字喔', '啊哈哈哈']): print(l)

ps = ckipnlp.parser.CkipParser(logger=False)
print(ps('中文字喔'))
for l in ps.apply_list(['中文字喔', '啊哈哈哈']): print(l)
ps.apply_file(ifile='sample/sample.txt', ofile='output/sample.tree')
with open('output/sample.tree') as fin:
    print(fin.read())

Utilities

import ckipnlp
print(ckipnlp.__name__, ckipnlp.__version__)

from ckipnlp.util.ws import *
from ckipnlp.util.parser import *

# Format CkipWs output
ws_text = ['中文字(Na)　喔(T)', '啊哈(I)　哈哈(D)']
for text in ws_text: print(ckipnlp.util.ws.WsSentence.from_text(text))
for text in ws_text: print(repr(ckipnlp.util.ws.WsSentence.from_text(text)))

# Show CkipParser output as tree
tree_text = 'S(theme:NP(property:N‧的(head:Nhaa:我|Head:DE:的)|Head:Nad(DUMMY1:Nab:早餐|Head:Caa:和|DUMMY2:Naa:午餐))|quantity:Dab:都|Head:VC31:吃完|aspect:Di:了)'
tree = ParserTree.from_text(tree_text)
tree.show()

# Get dummies of node 5
for node in tree.get_dummies(5): print(node)

# Get heads of node 1
for node in tree.get_heads(1): print(node)

# Get relations
for r in tree.get_relations(0): print(r)

FAQ

• The CKIPWS throws “what(): locale::facet::_S_create_c_locale name not valid”. What should I do?

apt-get install locales-all

License

• MIT License