c2.patch.jasplitter

This product is bugfix splitter of Plone for Japanese.

Monkey patching below functions.

Products.CMFPlone.UnicodeSplitter.splitter.bigram
Products.CMFPlone.UnicodeSplitter.splitter.process_unicode
Products.CMFPlone.UnicodeSplitter.splitter.process_unicode_glob

Details

bigram

return [u[i : i + 2] for i in range(len(u) - limit)]

if len(u) == 1:
    return [u]
else:
    return [u[i:i + 2] for i in range(len(u) - limit)]

process_unicode

swords = [g.group() for g in pattern.finditer(word)]
for sword in swords:
    if not rx_all.match(sword[0]):
        yield sword
    else:
        yield from bigram(sword, 0)

swords = [g.group() for g in pattern.finditer(word)]
for sword in swords:
    if not rx_all.match(sword[0]):
        yield sword
    else:
        for x in bigram(sword, 1):  # modified
            yield x

process_unicode_glob

if i == len(swords) - 1:
    limit = 1
else:
    limit = 0

limit = 1

Installation

Install c2.patch.jasplitter by adding it to your buildout:

[buildout]

...

eggs =
    c2.patch.jasplitter

and then running bin/buildout

Contribute

Issue Tracker: https://bitbucket.org/cmscom/c2.patch.jasplitter/admin/issues
Source Code: https://bitbucket.org/cmscom/c2.patch.jasplitter

Support

If you are having issues, please let us know on the issue tracker.

License

The project is licensed under the GPLv2.

Contributors

Manabu TERADA, terada@cmscom.jp

Changelog

1.0b1 (2024-02-06)

remove includeDependecies for plone6 compatibility. [terapyon]

1.0a5 (2023-08-24)

Upload PyPI. [terapyon]

1.0a4 (2023-08-17)

Support Python 3. [terapyon]

1.0a3 (2017-03-9)

Support continual CJK and Ascii words. [terapyon]
Missing packaging for MANIFEST [terapyon]

1.0a2 (2016-11-17)

Package bugfix. [terapyon]

1.0a1 (unreleased)

Initial release. [terapyon]

Metadata

Release files for c2.patch.jasplitter 1.0b1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for c2.patch.jasplitter 1.0b1
File	Size	Uploaded
c2.patch.jasplitter-1.0b1.tar.gz	14.2 kB	Feb 6, 2024	Details

Release files / c2.patch.jasplitter-1.0b1.tar.gz

Download URL	c2.patch.jasplitter-1.0b1.tar.gz
Size	14.2 kB
Tags	Source
SHA-256 checksum How to use checksums	`50fc39066c73d891f0a7a32307b6b10a3a5075ac85a1340b85d71a8ff2675ca6`
BLAKE2b-256 checksum How to use checksums	`85642bd28e415772e0e7d2d49a2b20067cae274969f768b060aa2595cedc4b3d`
Upload date	Feb 6, 2024
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/4.0.2 CPython/3.10.11`