suzume

Japanese tokenization for Python applications. Suzume is not a full morphological analyzer like MeCab: it prioritizes useful search units while still providing part-of-speech tags, lemmas, and keyword extraction. The wheel requires no extra dictionary installation.

📖 Full API reference and guides

See the MeCab comparison for concrete examples of token boundaries, lemmatization, and trade-offs.

Installation

pip install suzume

PyPI publishes binary wheels for Linux x86_64 (manylinux2014/manylinux_2_17) and macOS arm64 (macOS 11 or newer). Windows, macOS x86_64, Linux arm64, and other platforms or architectures are not supported. Suzume does not publish or support a source distribution; an installation succeeds only when a compatible wheel is available.

Quick Start

from suzume import Suzume

with Suzume() as sz:
    tokens = sz.analyze("すもももももももものうち")
    tags = sz.generate_tags("東京の公園に行きました")

Threads

Calls on one Suzume instance are serialized, so it can safely be shared between Python threads. Create one instance per worker when native analysis should run in parallel.

Installing the wheel also provides the suzume command:

suzume "東京へ行く"
suzume analyze --mode search --format json "東京の公園"
printf 'りんごを食べる\n' | suzume --format tags

Run suzume --help for analysis modes, normalization controls, output formats, tag filters, and repeatable --dict options. Dictionary compilation, validation, and test commands remain part of the separate native developer tool, suzume-cli.

See the Python API for the complete interface, including user dictionaries and tag-generation options.

Also available

npm install @libraz/suzume  # JavaScript / TypeScript WebAssembly package

The native C and C++ library is documented at suzume.libraz.net/docs/cpp.

License

Apache License 2.0