Skip to main content

python bindings of cppjieba

Project description

cppjieba-py

Build Status PyPI

cppjieba-py 是 cppjieba的 Python 封装。

由于只是对cppjieba的接口进行的封装,所以执行效率上非常接近于原cppjieba。

项目主要分为两个部分libcppjieba 为 cppjieba 的 python extension,
cppjieba_py 为使开发者平滑过渡到使用cppjieba-py而作的 python package。 具体见example.py

区别

  • 原jieba的.cut族接口基本都由python的iter()函数包裹list结果来实现。
  • 原jieba的.set_*方法基本都由class传入初始化的参数来完成。
  • .del_word.suggest_freq cppjieba没提供。
  • POSTokenizer.lcutTokenizer.tag 下, 唯一一个只提供了list返回类型的接口。

安装

  • pypi

    pip install cppjieba-py

    或者你设置的安装源并未收录本项目

    pip install -i https://pypi.org/simple/ cppjieba-py

  • 从发行包安装
    see releases

    pip install https://github.com/bung87/cppjieba-py/files/<xxxxxxx>/cppjieba_py-<x.x.x>.tar.gz

  • 从源代码安装

     $ git clone --recursive https://github.com/bung87/cppjieba-py
     $ python setup.py install 
    

使用

下面是一个使用 cppjieba-py 进行分词的例子

# -*- coding: utf-8 -*-
import cppjieba_py as jieba 
# or use defualt Tokenizer: jieba.cut 
jieba_instance = Tokenizer()
seg_list = jieba_instance.cut("我来到北京清华大学",cut_all = True)
print("Full Mode: " + "/ ".join(seg_list))  # 全模式


seg_list = jieba_instance.cut("他来到了网易杭研大厦")  # 默认是精确模式
print(", ".join(seg_list))

seg_list = jieba_instance.cut_for_search(
    "小明硕士毕业于中国科学院计算所,后在日本京都大学深造")  # 搜索引擎模式
print(", ".join(seg_list))
    

for more: example.py , tests

性能

performace_test/speed.py

测试方案:先按行读取文本围城到一个数组里,然后循环对围城每行文字作为一个句子进行分词。因为只对围城这本书分词一遍太快了,容易误差。 所以循环对围城这本书分词50次。基本上每次分词耗时都很稳定。 分词算法都是采用【精确模式】。

lcut HMM=False

方案 速度
cppjieba-py 10.642102
jieba-fast==0.51 26.129298
jieba==0.39 50.623866

lcut HMM=True

方案 速度
cppjieba-py 13.139232
jieba-fast==0.51 34.574907
jieba==0.39 1:26.756226

posseg.lcut

方案 速度
cppjieba-py 20.382905
jieba==0.39 1:19.411649

Test

pip install ".[test]"
nosetests -c nose.cfg

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

Built Distribution

File details

Details for the file cppjieba_py-0.0.13.macosx-10.14-x86_64.tar.gz.

File metadata

  • Download URL: cppjieba_py-0.0.13.macosx-10.14-x86_64.tar.gz
  • Upload date:
  • Size: 5.1 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.15.0 pkginfo/1.5.0.1 requests/2.18.4 setuptools/37.0.0 requests-toolbelt/0.8.0 tqdm/4.37.0 CPython/2.7.15

File hashes

Hashes for cppjieba_py-0.0.13.macosx-10.14-x86_64.tar.gz
Algorithm Hash digest
SHA256 39b834c2fa6bc853b0660d21fb87b1655ac019fad38a2e7aa6e060bf41d1fecc
MD5 e54ee98dbdae7f3aa721a20c9ab4475d
BLAKE2b-256 401a0a4a512453eb9b59535671fd32f889ce64f2f551ae83baafdca410cbb05d

See more details on using hashes here.

File details

Details for the file cppjieba_py-0.0.13-py3.5-macosx-10.14-x86_64.egg.

File metadata

  • Download URL: cppjieba_py-0.0.13-py3.5-macosx-10.14-x86_64.egg
  • Upload date:
  • Size: 5.2 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.15.0 pkginfo/1.5.0.1 requests/2.18.4 setuptools/37.0.0 requests-toolbelt/0.8.0 tqdm/4.37.0 CPython/2.7.15

File hashes

Hashes for cppjieba_py-0.0.13-py3.5-macosx-10.14-x86_64.egg
Algorithm Hash digest
SHA256 0bf7f4da16f0aaa7e91dd5bf0b9a14a6a31ba9d61461adbb9eac4df7887dc37d
MD5 7d3bc8378906ba42a52331efab9cff74
BLAKE2b-256 0f89936229995bb1ae3c9c62f91a74643e24eaf430c9b8db46cc7acac81a7f03

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page