kuzukiri
A simple text segmenter
What's this?
This is a python library for text segmentation of Japanese text.
Features
- Text segmentation by simple rules,
- rule-based, no machine learning,
- so you can assume results.
- comparably fast. It's written in rust-lang.
Install
from PyPI
pip install kuzukiri
from source code
pip install setuptools-rust
python setup.py install
Usage
import kuzukiri
segmenter = kuzukiri.Segmenter()
text = "これはテストです。文分割します。"
sentences = segmenter.split(text)
print(sentences) # => ['これはテストです。', '文分割します。']
For details, see examples and tests directories.
License
MIT
Dependencies
- PyO3 : to compile rust code for python.
- unicode_normalization crate : for NFKC normalization
1 maintainer
Metadata
1 maintainer
Release files for kuzukiri 0.1.3
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| kuzukiri-0.1.3.tar.gz | 5.0 kB | Details |
Built distributions (wheels)
| File | Reset | |||
|---|---|---|---|---|
| kuzukiri-0.1.3-cp310-cp310-manylinux1_x86_64.whl | CPython 3.10 | CPython 3.10 | Linux glibc 2.5+ x86-64 | Details |
| kuzukiri-0.1.3-cp310-cp310-macosx_12_0_x86_64.whl | CPython 3.10 | CPython 3.10 | macOS 12.0+ x86-64 | Details |
| kuzukiri-0.1.3-cp39-cp39-manylinux1_x86_64.whl | CPython 3.9 | CPython 3.9 | Linux glibc 2.5+ x86-64 | Details |
| kuzukiri-0.1.3-cp39-cp39-macosx_12_0_x86_64.whl | CPython 3.9 | CPython 3.9 | macOS 12.0+ x86-64 | Details |
| kuzukiri-0.1.3-cp38-cp38-manylinux1_x86_64.whl | CPython 3.8 | CPython 3.8 | Linux glibc 2.5+ x86-64 | Details |
| kuzukiri-0.1.3-cp38-cp38-macosx_12_0_x86_64.whl | CPython 3.8 | CPython 3.8 | macOS 12.0+ x86-64 | Details |
| kuzukiri-0.1.3-cp37-cp37m-manylinux1_x86_64.whl | CPython 3.7 | CPython 3.7 pymalloc | Linux glibc 2.5+ x86-64 | Details |
| kuzukiri-0.1.3-cp37-cp37m-macosx_12_0_x86_64.whl | CPython 3.7 | CPython 3.7 pymalloc | macOS 12.0+ x86-64 | Details |
Total release size: 5.6 MB
Release files / kuzukiri-0.1.3.tar.gz
| Download URL | kuzukiri-0.1.3.tar.gz |
|---|---|
| Size | 5.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
c8c5ff9c892086f0e7eab02e3d7bb3546ba2b1683bab0d70f6e3f5b96378e850
|
|
BLAKE2b-256 checksum How to use checksums |
81387200f22d50acb382faec99ee2c99d536725142c9f1ba4198bfa03fb4e65a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.7.1 importlib_metadata/4.12.0 pkginfo/1.8.3 requests/2.28.1 requests-toolbelt/0.9.1 tqdm/4.64.1 CPython/3.8.10
|
Release files / kuzukiri-0.1.3-cp310-cp310-manylinux1_x86_64.whl
| Download URL | kuzukiri-0.1.3-cp310-cp310-manylinux1_x86_64.whl |
|---|---|
| Size | 1.1 MB |
| Tags | CPython 3.10 Linux glibc 2.5+ x86-64 |
|
SHA-256 checksum How to use checksums |
1d4c7bf5d7d20654bfa2522753ee3794d25139f42bc3a321182f713625024398
|
|
BLAKE2b-256 checksum How to use checksums |
7b3ab0ca1d5c0f649812d3f70f7f66cad3ca96f6e811fae1c979f5f53c2236b5
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.7.1 importlib_metadata/4.12.0 pkginfo/1.8.3 requests/2.28.1 requests-toolbelt/0.9.1 tqdm/4.64.1 CPython/3.10.7
|
Release files / kuzukiri-0.1.3-cp310-cp310-macosx_12_0_x86_64.whl
| Download URL | kuzukiri-0.1.3-cp310-cp310-macosx_12_0_x86_64.whl |
|---|---|
| Size | 290.8 kB |
| Tags | CPython 3.10 macOS 12.0+ x86-64 |
|
SHA-256 checksum How to use checksums |
b022424f16e2d3115f7bbe946c02821b4432e1b44b12750f2395388448e64f3d
|
|
BLAKE2b-256 checksum How to use checksums |
1655bfcc54eef29d56f8430673e3fab41bacb3265302277dd3a1fcaafde2b899
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.7.1 importlib_metadata/4.12.0 pkginfo/1.8.3 requests/2.28.1 requests-toolbelt/0.9.1 tqdm/4.64.1 CPython/3.10.6
|
Release files / kuzukiri-0.1.3-cp39-cp39-manylinux1_x86_64.whl
| Download URL | kuzukiri-0.1.3-cp39-cp39-manylinux1_x86_64.whl |
|---|---|
| Size | 1.1 MB |
| Tags | CPython 3.9 Linux glibc 2.5+ x86-64 |
|
SHA-256 checksum How to use checksums |
3c7fc6a5b105ce7f82f4bff607bbe1dad847293223424c72906b51c67b70d603
|
|
BLAKE2b-256 checksum How to use checksums |
6865fdaad9f712d41511c84b942de7c86a181e93702ea68fa443a9d4b6a5a49b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.7.1 importlib_metadata/4.12.0 pkginfo/1.8.3 requests/2.28.1 requests-toolbelt/0.9.1 tqdm/4.64.1 CPython/3.9.5
|
Release files / kuzukiri-0.1.3-cp39-cp39-macosx_12_0_x86_64.whl
| Download URL | kuzukiri-0.1.3-cp39-cp39-macosx_12_0_x86_64.whl |
|---|---|
| Size | 290.9 kB |
| Tags | CPython 3.9 macOS 12.0+ x86-64 |
|
SHA-256 checksum How to use checksums |
70775db3cb275d0a29c539819ae2ac61c2db8f784a34a4b284ff9856a54ef6e9
|
|
BLAKE2b-256 checksum How to use checksums |
0164ca6153fc094753746aa58f7b85bf81e848ccd1c57a089054724136a50425
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.7.1 importlib_metadata/4.12.0 pkginfo/1.8.3 requests/2.28.1 requests-toolbelt/0.9.1 tqdm/4.64.1 CPython/3.9.14
|
Release files / kuzukiri-0.1.3-cp38-cp38-manylinux1_x86_64.whl
| Download URL | kuzukiri-0.1.3-cp38-cp38-manylinux1_x86_64.whl |
|---|---|
| Size | 1.1 MB |
| Tags | CPython 3.8 Linux glibc 2.5+ x86-64 |
|
SHA-256 checksum How to use checksums |
f2a129e38d4684a29b0eaf6d0e5601a363e460f7943a701dce619c26b38e470f
|
|
BLAKE2b-256 checksum How to use checksums |
b047c10792e27c434e9ac36493d8d655e3232fa245585c42f0cbbf351244d06d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.7.1 importlib_metadata/4.12.0 pkginfo/1.8.3 requests/2.28.1 requests-toolbelt/0.9.1 tqdm/4.64.1 CPython/3.8.10
|
Release files / kuzukiri-0.1.3-cp38-cp38-macosx_12_0_x86_64.whl
| Download URL | kuzukiri-0.1.3-cp38-cp38-macosx_12_0_x86_64.whl |
|---|---|
| Size | 291.0 kB |
| Tags | CPython 3.8 macOS 12.0+ x86-64 |
|
SHA-256 checksum How to use checksums |
e38bfda84ec090b89654308f86e27f57a50c924e6a468cfb2b16edd068b923f4
|
|
BLAKE2b-256 checksum How to use checksums |
d1ceaf5a65d31c6de709f3d2e70df052d7065ed0d8036472e6f3d1704b7bbe45
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.7.1 importlib_metadata/4.12.0 pkginfo/1.8.3 requests/2.28.1 requests-toolbelt/0.9.1 tqdm/4.64.1 CPython/3.8.14
|
Release files / kuzukiri-0.1.3-cp37-cp37m-manylinux1_x86_64.whl
| Download URL | kuzukiri-0.1.3-cp37-cp37m-manylinux1_x86_64.whl |
|---|---|
| Size | 1.1 MB |
| Tags | CPython 3.7 CPython 3.7 pymalloc Linux glibc 2.5+ x86-64 |
|
SHA-256 checksum How to use checksums |
61791df3efa53affeefcef54fbdbf48c193f4a7ce25aee6b8c965cab027febb9
|
|
BLAKE2b-256 checksum How to use checksums |
d3355a6031b8f8841d5fb4561f845cef17ce4ec270d76305ddf70700d7b4c2e4
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.7.1 importlib_metadata/4.12.0 pkginfo/1.8.3 requests/2.28.1 requests-toolbelt/0.9.1 tqdm/4.64.1 CPython/3.7.13
|
Release files / kuzukiri-0.1.3-cp37-cp37m-macosx_12_0_x86_64.whl
| Download URL | kuzukiri-0.1.3-cp37-cp37m-macosx_12_0_x86_64.whl |
|---|---|
| Size | 291.2 kB |
| Tags | CPython 3.7 CPython 3.7 pymalloc macOS 12.0+ x86-64 |
|
SHA-256 checksum How to use checksums |
cb109853d3f6910bc9ea2ff6ee8585db4eade12f793eed994d932f64b8d5e6a2
|
|
BLAKE2b-256 checksum How to use checksums |
9081e2fe46787884dfc0f84e8432f4c8c3dabf593748da06c599049c6417adb3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.7.1 importlib_metadata/4.12.0 pkginfo/1.8.3 requests/2.28.1 requests-toolbelt/0.9.1 tqdm/4.64.1 CPython/3.7.14
|
