suzume 0.9.5

Lightweight dictionary-independent Japanese morphological analyzer

suzume

Lightweight, dictionary-independent Japanese morphological analyzer (tokenizer + part-of-speech tagging). This package is a thin ctypes binding over the native suzume C++ core; the compiled library and dictionaries are bundled in the wheel, so there is nothing else to install.

Installation

pip install suzume

Quick start

from suzume import Suzume

with Suzume() as sz:
    for m in sz.analyze("東京都に住む"):
        print(m.surface, m.pos, m.base_form)

Generate keyword tags:

from suzume import Suzume

with Suzume() as sz:
    for tag in sz.generate_tags("東京都に住んでいます"):
        print(tag.tag, tag.pos)

    # Restrict to selected parts of speech by name (or a raw bitmask).
    nouns = sz.generate_tags("美味しいラーメンを食べた", pos_filter=["noun"])

API

Suzume(*, mode="normal", preserve_vu=True, preserve_case=True, preserve_symbols=False, lemmatize=True, merge_compounds=False)

An analyzer instance. Use it as a context manager (or call close()); it is not thread-safe, so use one instance per thread.

• mode — "normal", "search", or "split" (or a Mode enum member).
• analyze(text) -> list[Morpheme] — tokenize with full POS information.
• generate_tags(text, **options) -> list[Tag] — extract keyword tags. The pos_filter option accepts a raw bitmask (1=noun, 2=verb, 4=adjective, 8=adverb) or an iterable of POS names such as ["noun", "verb"].
• load_user_dict(csv) — add a user dictionary from CSV text.
• load_binary_dict(data) — load a compiled .dic dictionary from memory.
• dictionary_warnings — warnings raised while auto-loading dictionaries.

Morpheme

A frozen dataclass with surface, pos, base_form, pos_ja, conj_type, conj_form, extended_pos, start, end, the is_* flags, and score. conj_type and conj_form are None for non-conjugating words.

Tag

A frozen dataclass with tag and pos.

suzume.version()

Returns the native library version string.

License

Apache-2.0. See LICENSE.