Weblate is libre software web-based continuous localization system, used by over 2500 libre projects and companies in more than 165 countries.

unicode-segmentation-rs

Python bindings for the Rust unicode-segmentation, unicode-linebreak, and unicode-width crates, providing Unicode text segmentation and width calculation according to Unicode standards.

Features

Grapheme Cluster Segmentation: Split text into user-perceived characters
Word Segmentation: Split text into words according to Unicode rules
Sentence Segmentation: Split text into sentences
Line-Break Segmentation: Find legal wrapping opportunities according to Unicode Standard Annex #14
Display Width Calculation: Get the display width of text (for terminal/monospace display)
Gettext PO Wrapping: Wrap text for gettext PO files with proper handling of escape sequences and CJK characters

Installation

From PyPI

uv pip install unicode-segmentation-rs

From source

# Install maturin
pip install maturin

# Build and install the package
maturin develop --release

Usage

import unicode_segmentation_rs

# Grapheme clusters (user-perceived characters)
text = "Hello 👨‍👩‍👧‍👦 World"
clusters = unicode_segmentation_rs.graphemes(text, is_extended=True)
print(clusters)  # ['H', 'e', 'l', 'l', 'o', ' ', '👨‍👩‍👧‍👦', ' ', 'W', 'o', 'r', 'l', 'd']

# Get grapheme clusters with their byte indices
indices = unicode_segmentation_rs.grapheme_indices(text, is_extended=True)
print(indices)  # [(0, 'H'), (1, 'e'), ...]

# Word boundaries (includes punctuation and whitespace)
text = "Hello, world!"
words = unicode_segmentation_rs.split_word_bounds(text)
print(words)  # ['Hello', ',', ' ', 'world', '!']

# Unicode words (excludes punctuation and whitespace)
words = unicode_segmentation_rs.unicode_words(text)
print(words)  # ['Hello', 'world']

# Word indices
indices = unicode_segmentation_rs.split_word_bound_indices(text)
print(indices)  # [(0, 'Hello'), (5, ','), ...]

# Sentence segmentation
text = "Hello world. How are you? I'm fine."
sentences = unicode_segmentation_rs.unicode_sentences(text)
print(sentences)  # ['Hello world. ', 'How are you? ', "I'm fine."]

# Line-break opportunities use UTF-8 byte offsets
text = "你好世界"
breaks = unicode_segmentation_rs.line_breaks(text)
print(breaks)  # [(3, False), (6, False), (9, False), (12, True)]
units = unicode_segmentation_rs.line_break_units(text)
print(units)  # ['你', '好', '世', '界']

# Display width calculation
text = "Hello 世界"
width = unicode_segmentation_rs.text_width(text)
print(width)  # 10 (Hello=5, space=1, 世=2, 界=2, but depends on terminal)

# Character width
print(unicode_segmentation_rs.text_width('A'))    # 1
print(unicode_segmentation_rs.text_width('世'))   # 2
print(unicode_segmentation_rs.text_width('\t'))   # 1

Examples

Grapheme Cluster Segmentation

import unicode_segmentation_rs

# Complex emojis and combining characters
text = "Hello 👨‍👩‍👧‍👦 नमस्ते"
print(f"Text: {text}")
print(f"Graphemes: {unicode_segmentation_rs.graphemes(text, is_extended=True)}")
print(f"Length (graphemes): {len(unicode_segmentation_rs.graphemes(text, is_extended=True))}")
print(f"Length (chars): {len(text)}")

# With indices
print("Grapheme indices:")
for idx, cluster in unicode_segmentation_rs.grapheme_indices(text, is_extended=True):
    print(f"  {idx:3d}: {cluster!r}")

Word Segmentation

text = "Hello, world! How are you?"
print(f"Text: {text}")
print(f"Word bounds: {unicode_segmentation_rs.split_word_bounds(text)}")
print(f"Unicode words: {unicode_segmentation_rs.unicode_words(text)}")

# With indices
print("Word boundary indices:")
for idx, word in unicode_segmentation_rs.split_word_bound_indices(text):
    print(f"  {idx:3d}: {word!r}")

Sentence Segmentation

text = "Hello world. How are you? I'm fine, thanks! What about you?"
print(f"Text: {text}")
sentences = unicode_segmentation_rs.unicode_sentences(text)
print("Sentences:")
for i, sentence in enumerate(sentences, 1):
    print(f"  {i}. {sentence!r}")

Line-Break Segmentation

# Find legal wrapping opportunities without measuring or reformatting text
text = "Hello 世界"
print(unicode_segmentation_rs.line_breaks(text))
print(unicode_segmentation_rs.line_break_units(text))  # ['Hello ', '世', '界']

# A soft hyphen is preserved in the unit before its discretionary break
text = "foo\u00adbar"
print(unicode_segmentation_rs.line_break_units(text))  # ['foo\u00ad', 'bar']

Line-break segmentation reports opportunities only. The caller remains responsible for measuring widths, trimming whitespace for display, and rendering a visible hyphen when selecting a soft-hyphen break.

The underlying unicode-linebreak crate treats complex-context (SA) characters as ordinary alphabetic characters rather than performing dictionary-based segmentation. Consequently, scripts such as Thai, Lao, and Khmer do not receive language-specific word break opportunities.

Multilingual Examples

# Arabic
arabic = "مرحبا بك. كيف حالك؟"
print(f"Arabic: {arabic}")
print(f"Sentences: {unicode_segmentation_rs.unicode_sentences(arabic)}")

# Japanese
japanese = "こんにちは。お元気ですか？"
print(f"Japanese: {japanese}")
print(f"Sentences: {unicode_segmentation_rs.unicode_sentences(japanese)}")

# Mixed languages
mixed = "Hello世界! This is a test文章."
print(f"Mixed: {mixed}")
print(f"Words: {unicode_segmentation_rs.unicode_words(mixed)}")

Display Width Calculation

examples = [
    "Hello",
    "世界",
    "Hello 世界",
    "こんにちは",
    "🎉🎊",
    "Tab\there",
]

for text in examples:
    width = unicode_segmentation_rs.text_width(text)
    print(f"Text: {text!r:20} Width: {width:2} Chars: {len(text):2}")

# Character widths
chars = ['a', 'A', '1', ' ', '世', '界', 'あ', '🎉', '\t', '\n']
for c in chars:
    w = unicode_segmentation_rs.text_width(c)
    print(f"  {c!r:6} width: {w:2}")

Gettext PO File Wrapping

# Wrap text for PO files (default width is 77 characters)
text = "This is a long translation string that needs to be wrapped appropriately for a gettext PO file"
lines = unicode_segmentation_rs.gettext_wrap(text, 77)
for i, line in enumerate(lines, 1):
    print(f"Line {i}: {line}")

# Wrapping with CJK characters
text = "This translation contains 中文字符 (Chinese characters) and should wrap correctly"
lines = unicode_segmentation_rs.gettext_wrap(text, 40)
for line in lines:
    width = unicode_segmentation_rs.text_width(line)
    print(f"[{width:2d} cols] {line}")

# Escape sequences are preserved
text = "This has\\nline breaks\\tand tabs"
lines = unicode_segmentation_rs.gettext_wrap(text, 20)
print(lines)

API Reference

Functions returning lists materialize the complete result. Applications processing very large or untrusted text should enforce an input-size limit appropriate for their environment.

`graphemes(text: str, is_extended: bool) -> list[str]`

Split a string into grapheme clusters. Set is_extended=True for extended grapheme clusters (recommended).

`grapheme_indices(text: str, is_extended: bool) -> list[tuple[int, str]]`

Split a string into grapheme clusters with their byte indices.

`split_word_bounds(text: str) -> list[str]`

Split a string at word boundaries (includes punctuation and whitespace).

`split_word_bound_indices(text: str) -> list[tuple[int, str]]`

Split a string at word boundaries with byte indices.

`unicode_words(text: str) -> list[str]`

Get Unicode words from a string (excludes punctuation and whitespace).

`unicode_sentences(text: str) -> list[str]`

Split a string into sentences according to Unicode rules.

`line_breaks(text: str) -> list[tuple[int, bool]]`

Return every allowed or mandatory Unicode line-break opportunity as (utf8_byte_offset, mandatory). The boolean is True only for mandatory breaks. Non-empty input includes the mandatory end-of-text opportunity.

`line_break_units(text: str) -> list[str]`

Split a string at every allowed or mandatory Unicode line-break opportunity. Every input code point is preserved exactly, and the mandatory end-of-text opportunity does not add an empty unit.

`text_width(text: str) -> int`

Get the display width of a string in columns (as it would appear in a terminal). East Asian characters typically take 2 columns.

`gettext_wrap(text: str, width: int) -> list[str]`

Wrap text for gettext PO files. This function follows gettext's wrapping behavior:

Never breaks escape sequences (\n, \", etc.)
Prefers breaking after spaces
Handles CJK characters with proper width calculation
Breaks long words only when necessary

Building for Distribution

# Build wheel
maturin build --release

# Build and publish to PyPI
maturin publish

Running Tests

# Install test dependencies
pip install pytest

# Run tests
pytest tests/

License

This project is licensed under the Apache License 2.0.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

unicode_segmentation_rs-0.3.3.tar.gz (34.8 kB view details)

Uploaded Aug 8, 2026 Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

unicode_segmentation_rs-0.3.3-cp315-abi3.abi3t-win_amd64.whl (165.8 kB view details)

Uploaded Aug 8, 2026 CPython 3.15CPython 3.15+Windows x86-64

unicode_segmentation_rs-0.3.3-cp315-abi3.abi3t-win32.whl (161.0 kB view details)

Uploaded Aug 8, 2026 CPython 3.15CPython 3.15+Windows x86

unicode_segmentation_rs-0.3.3-cp315-abi3.abi3t-musllinux_1_2_x86_64.whl (506.3 kB view details)

Uploaded Aug 8, 2026 CPython 3.15CPython 3.15+musllinux: musl 1.2+ x86-64

unicode_segmentation_rs-0.3.3-cp315-abi3.abi3t-musllinux_1_2_i686.whl (538.7 kB view details)

Uploaded Aug 8, 2026 CPython 3.15CPython 3.15+musllinux: musl 1.2+ i686

unicode_segmentation_rs-0.3.3-cp315-abi3.abi3t-musllinux_1_2_armv7l.whl (578.6 kB view details)

Uploaded Aug 8, 2026 CPython 3.15CPython 3.15+musllinux: musl 1.2+ ARMv7l

unicode_segmentation_rs-0.3.3-cp315-abi3.abi3t-musllinux_1_2_aarch64.whl (475.3 kB view details)

Uploaded Aug 8, 2026 CPython 3.15CPython 3.15+musllinux: musl 1.2+ ARM64

unicode_segmentation_rs-0.3.3-cp315-abi3.abi3t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (303.6 kB view details)

Uploaded Aug 8, 2026 CPython 3.15CPython 3.15+manylinux: glibc 2.17+ x86-64

unicode_segmentation_rs-0.3.3-cp315-abi3.abi3t-manylinux_2_17_s390x.manylinux2014_s390x.whl (317.7 kB view details)

Uploaded Aug 8, 2026 CPython 3.15CPython 3.15+manylinux: glibc 2.17+ s390x

unicode_segmentation_rs-0.3.3-cp315-abi3.abi3t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (420.3 kB view details)

Uploaded Aug 8, 2026 CPython 3.15CPython 3.15+manylinux: glibc 2.17+ ppc64le

unicode_segmentation_rs-0.3.3-cp315-abi3.abi3t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (303.4 kB view details)

Uploaded Aug 8, 2026 CPython 3.15CPython 3.15+manylinux: glibc 2.17+ ARMv7l

unicode_segmentation_rs-0.3.3-cp315-abi3.abi3t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (297.1 kB view details)

Uploaded Aug 8, 2026 CPython 3.15CPython 3.15+manylinux: glibc 2.17+ ARM64

unicode_segmentation_rs-0.3.3-cp315-abi3.abi3t-manylinux_2_5_i686.manylinux1_i686.whl (318.1 kB view details)

Uploaded Aug 8, 2026 CPython 3.15CPython 3.15+manylinux: glibc 2.5+ i686

unicode_segmentation_rs-0.3.3-cp315-abi3.abi3t-macosx_11_0_arm64.whl (270.8 kB view details)

Uploaded Aug 8, 2026 CPython 3.15CPython 3.15+macOS 11.0+ ARM64

unicode_segmentation_rs-0.3.3-cp315-abi3.abi3t-macosx_10_12_x86_64.whl (271.5 kB view details)

Uploaded Aug 8, 2026 CPython 3.15CPython 3.15+macOS 10.12+ x86-64

unicode_segmentation_rs-0.3.3-cp314-cp314t-win_amd64.whl (165.9 kB view details)

Uploaded Aug 8, 2026 CPython 3.14tWindows x86-64

unicode_segmentation_rs-0.3.3-cp314-cp314t-win32.whl (161.2 kB view details)

Uploaded Aug 8, 2026 CPython 3.14tWindows x86

unicode_segmentation_rs-0.3.3-cp314-cp314t-musllinux_1_2_x86_64.whl (506.7 kB view details)

Uploaded Aug 8, 2026 CPython 3.14tmusllinux: musl 1.2+ x86-64

unicode_segmentation_rs-0.3.3-cp314-cp314t-musllinux_1_2_i686.whl (538.9 kB view details)

Uploaded Aug 8, 2026 CPython 3.14tmusllinux: musl 1.2+ i686

unicode_segmentation_rs-0.3.3-cp314-cp314t-musllinux_1_2_armv7l.whl (579.2 kB view details)

Uploaded Aug 8, 2026 CPython 3.14tmusllinux: musl 1.2+ ARMv7l

unicode_segmentation_rs-0.3.3-cp314-cp314t-musllinux_1_2_aarch64.whl (475.6 kB view details)

Uploaded Aug 8, 2026 CPython 3.14tmusllinux: musl 1.2+ ARM64

unicode_segmentation_rs-0.3.3-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (303.4 kB view details)

Uploaded Aug 8, 2026 CPython 3.14tmanylinux: glibc 2.17+ x86-64

unicode_segmentation_rs-0.3.3-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl (317.6 kB view details)

Uploaded Aug 8, 2026 CPython 3.14tmanylinux: glibc 2.17+ s390x

unicode_segmentation_rs-0.3.3-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (419.7 kB view details)

Uploaded Aug 8, 2026 CPython 3.14tmanylinux: glibc 2.17+ ppc64le

unicode_segmentation_rs-0.3.3-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (303.9 kB view details)

Uploaded Aug 8, 2026 CPython 3.14tmanylinux: glibc 2.17+ ARMv7l

unicode_segmentation_rs-0.3.3-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (297.6 kB view details)

Uploaded Aug 8, 2026 CPython 3.14tmanylinux: glibc 2.17+ ARM64

unicode_segmentation_rs-0.3.3-cp314-cp314t-manylinux_2_5_i686.manylinux1_i686.whl (318.6 kB view details)

Uploaded Aug 8, 2026 CPython 3.14tmanylinux: glibc 2.5+ i686

unicode_segmentation_rs-0.3.3-cp314-cp314t-macosx_11_0_arm64.whl (271.2 kB view details)

Uploaded Aug 8, 2026 CPython 3.14tmacOS 11.0+ ARM64

unicode_segmentation_rs-0.3.3-cp314-cp314t-macosx_10_12_x86_64.whl (271.8 kB view details)

Uploaded Aug 8, 2026 CPython 3.14tmacOS 10.12+ x86-64

unicode_segmentation_rs-0.3.3-cp310-abi3-win_amd64.whl (168.7 kB view details)

Uploaded Aug 8, 2026 CPython 3.10+Windows x86-64

unicode_segmentation_rs-0.3.3-cp310-abi3-win32.whl (163.7 kB view details)

Uploaded Aug 8, 2026 CPython 3.10+Windows x86

unicode_segmentation_rs-0.3.3-cp310-abi3-musllinux_1_2_x86_64.whl (510.0 kB view details)

Uploaded Aug 8, 2026 CPython 3.10+musllinux: musl 1.2+ x86-64

unicode_segmentation_rs-0.3.3-cp310-abi3-musllinux_1_2_i686.whl (542.7 kB view details)

Uploaded Aug 8, 2026 CPython 3.10+musllinux: musl 1.2+ i686

unicode_segmentation_rs-0.3.3-cp310-abi3-musllinux_1_2_armv7l.whl (582.0 kB view details)

Uploaded Aug 8, 2026 CPython 3.10+musllinux: musl 1.2+ ARMv7l

unicode_segmentation_rs-0.3.3-cp310-abi3-musllinux_1_2_aarch64.whl (479.7 kB view details)

Uploaded Aug 8, 2026 CPython 3.10+musllinux: musl 1.2+ ARM64

unicode_segmentation_rs-0.3.3-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (307.4 kB view details)

Uploaded Aug 8, 2026 CPython 3.10+manylinux: glibc 2.17+ x86-64

unicode_segmentation_rs-0.3.3-cp310-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl (320.0 kB view details)

Uploaded Aug 8, 2026 CPython 3.10+manylinux: glibc 2.17+ s390x

unicode_segmentation_rs-0.3.3-cp310-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (424.0 kB view details)

Uploaded Aug 8, 2026 CPython 3.10+manylinux: glibc 2.17+ ppc64le

unicode_segmentation_rs-0.3.3-cp310-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (306.6 kB view details)

Uploaded Aug 8, 2026 CPython 3.10+manylinux: glibc 2.17+ ARMv7l

unicode_segmentation_rs-0.3.3-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (301.6 kB view details)

Uploaded Aug 8, 2026 CPython 3.10+manylinux: glibc 2.17+ ARM64

unicode_segmentation_rs-0.3.3-cp310-abi3-manylinux_2_5_i686.manylinux1_i686.whl (322.8 kB view details)

Uploaded Aug 8, 2026 CPython 3.10+manylinux: glibc 2.5+ i686

unicode_segmentation_rs-0.3.3-cp310-abi3-macosx_11_0_arm64.whl (275.4 kB view details)

Uploaded Aug 8, 2026 CPython 3.10+macOS 11.0+ ARM64

unicode_segmentation_rs-0.3.3-cp310-abi3-macosx_10_12_x86_64.whl (273.8 kB view details)

Uploaded Aug 8, 2026 CPython 3.10+macOS 10.12+ x86-64

File details

Details for the file unicode_segmentation_rs-0.3.3.tar.gz.

File metadata

Download URL: unicode_segmentation_rs-0.3.3.tar.gz
Upload date: Aug 8, 2026
Size: 34.8 kB
Tags: Source
Uploaded using Trusted Publishing? Yes
Uploaded via: maturin/1.14.1

File hashes

Hashes for unicode_segmentation_rs-0.3.3.tar.gz
Algorithm	Hash digest
SHA256	`d6625b2d3435ca814c9dd6590d39ae58ebeb8a4891eecb81446ad8b3e917f39b`
MD5	`ca0a9d21e808864cf0dfca81d9d4df25`
BLAKE2b-256	`0b02e5804acc54945ecf29a280f5f173db61c019166bfe3adeee386f4c135f17`

Algorithm	Hash digest
SHA256	`e785d350bca7c8ed999988d42a695ac191a0465b475c0a37b07a313470fe53be`
MD5	`6c218021d4bf7d9334ce28bc237f5ff1`
BLAKE2b-256	`8393309740e880ad491c6ca5ec581829bd27525d94a30b71678fa0a202bc589c`

Algorithm	Hash digest
SHA256	`5aa406cb294c4876af6f91af6b0d3a0a689187f4bc59310f7df13570bc119fe2`
MD5	`556f21dceedaba29f0d7e72c585527dd`
BLAKE2b-256	`9475b25d9431aaa63c75e1806b4b65f0c676fa776510258713ff8405424ff845`

Algorithm	Hash digest
SHA256	`342b29c9ec9e7012a43cfb9166c2cacbddb5b3135cbdc2c0886842e6deaa9f99`
MD5	`900f5d968765dae638534a57c0829951`
BLAKE2b-256	`52124c6a095387f39e4b3b90b5f601c06fb05608251b1afdc0cd9d389908f1b5`

Algorithm	Hash digest
SHA256	`6f075eab90534430522c392e908b5b9df8eaaab7c2f906a3f462f5885d32c0f8`
MD5	`34e8ea4e15cccab7fdf303f31e707c57`
BLAKE2b-256	`b748bc2b75616fb2080a77897b84deee245c3762b960bfc4621124dce9abe6ee`

Algorithm	Hash digest
SHA256	`1cba91dc74b4c47099fc08b5dcf5fb1164fc56a12eb08417a3705fb380154f14`
MD5	`9cff94451ec8a9cb872d4dcc70dc840d`
BLAKE2b-256	`63653848035d97790e8a897a8f56caaed00e2df14256cfaab18c03c76487062b`

Algorithm	Hash digest
SHA256	`8dcc42d0d0b6b86654dac774ec438dd34a71e89a32c9d6b70791d95f2073f480`
MD5	`17efbc617c16671710bdef87367b0b50`
BLAKE2b-256	`ee1e5be4457ae25e4304e06d5101d00dabb8ad026a8781b59afe7ba276f9bf04`

Algorithm	Hash digest
SHA256	`8f0d1c7620f256905ecc7103370160f4cd1f75144a1f41344b8e9b62550ab692`
MD5	`0041f29e75640f5d2b669c38ea376000`
BLAKE2b-256	`0005275f5a0d45b5ac7d6b4553b15d1adec785329f94b255116236411e369796`

Algorithm	Hash digest
SHA256	`0158bc51223110a2efd9b1235fb79687e4945217abdd8a53f4708499379c9a07`
MD5	`be99cd2796d5448fef7834f1ee1404bb`
BLAKE2b-256	`4506c3d45f08933d62b2f0db58c35e0036cfa8484cbf204fd707a933316171ac`

Algorithm	Hash digest
SHA256	`2df52306912976755b9bdba2eab224be7bf02c399b46f1fc10fd9aa514d80f57`
MD5	`9c00b89883a914c684dfe3198f4b7334`
BLAKE2b-256	`6ff3c0574c2d71ccd4946cf12a61a1f789287f8067b6d73339a4992166f4e975`

Algorithm	Hash digest
SHA256	`a1a59d8b81b7685a216e0586cfc27d8615fa4007e5a8eee81f77f74a192364e2`
MD5	`96fb3249693cd3d7d123166ada9d453f`
BLAKE2b-256	`22d2c9d8adb7c5dd73b8e38889d38f0075d60ea61bd4e8af551923772ed7ac7e`

Algorithm	Hash digest
SHA256	`ec4129eaeb8267103e0bd56efa675f450493c8f99c826367df9a71747d9e6d06`
MD5	`2e3cd1313c96cf49ec68eb98541a5ffb`
BLAKE2b-256	`1d938806474bd027c2b8db7b9f5806933985d11e95ca2b90c73e44a03413cac8`

Algorithm	Hash digest
SHA256	`fcc0758cd1d8e33ed8a65bec1508c7ff74c5fab4a35f2bf7b65983723c3e34b0`
MD5	`bb499b33fcf78668c3e6c26a3892f63e`
BLAKE2b-256	`7455abab41a447d4de268b35ab8362e085572077680bdeb151471b4b0e10bc79`

Algorithm	Hash digest
SHA256	`36aaf0d129a22527ceaabb8587b0253fbd7950c4e5bf5f3a5bdcb4f330d30133`
MD5	`7e247256362e003c89d195fa02777982`
BLAKE2b-256	`eee7827a8821ca259c0cc5df9dd5b42b28164ee9feab094ec91b863a8e906e75`

Algorithm	Hash digest
SHA256	`c2fce9d3d003c4ef1c96c1e84d4f0b4cdd75c5c9ea2bb8aa1bf3ac796bc1dee0`
MD5	`781f64f264cbcbc39a761130a9bac1a3`
BLAKE2b-256	`6ee32175394ed0fab546bd0dd48dca5bdf98384052482609a289bd43592f0b6e`

Algorithm	Hash digest
SHA256	`4768ed7d90795727341222f02169a56b43115adbe69f0e91e3af4133c0f93c30`
MD5	`90ae3f297b2fa16b0ecf3c046271305e`
BLAKE2b-256	`86635c588c01f7a4902c859af768a80d4801186844da30cdd46beadaae0a6cb3`

Algorithm	Hash digest
SHA256	`40158b7da74ab02419b071bbb73204e64790c78cdfc1d403d8549290bae835a2`
MD5	`916c645e18101f0bdedbd5aa888229be`
BLAKE2b-256	`2ed4878004bc6c732a2d5c5a7eec755ddc489cd29fca0cf2d5b04c1576f60158`

Algorithm	Hash digest
SHA256	`d8a3522ef2bd26052e3cb33c341995aeb46ac87d57d1a5438935e9c4cf2f9056`
MD5	`dd02267daabc375392d6c58e20bb7e36`
BLAKE2b-256	`3691686adaa8af63080df15ad903aa6aa63591353122f74b7996aa71861b7e26`

Algorithm	Hash digest
SHA256	`387594f6005ed0b7b7e4d778fd953a476628f2e8ce8335ca167fef6140f1c498`
MD5	`c8339240a46c1e081b38ccc4a82b6b32`
BLAKE2b-256	`63851fc203e68bf8a808f93fc877054ae0d65854d63a0a9e6e0e966f5032c675`

Algorithm	Hash digest
SHA256	`63de5bceca7f0dc753454dcf14e88e932dd51e8a6f8620a8183ceb613dc68db8`
MD5	`a06c960e59e3f15f0d22846e939a40eb`
BLAKE2b-256	`fe83283845679c91064744c7f7806108f12f90a614c23a37c6a43657249c818f`

Algorithm	Hash digest
SHA256	`3012d68e6d806b317571f7e1e6969930c150bf380c39d670b98cd524c4b2f35d`
MD5	`8f89e47baa5fb710f285eb36f2d62453`
BLAKE2b-256	`cffc14b8dfd7ecb652945de1397855d65e84ff1a3f035521a4067fa455d5c49a`

unicode-segmentation-rs 0.3.3

unicode-segmentation-rs

Features

Installation

From PyPI

From source

Usage

Examples

Grapheme Cluster Segmentation

Word Segmentation

Sentence Segmentation

Line-Break Segmentation

Multilingual Examples

Display Width Calculation

Gettext PO File Wrapping

API Reference

graphemes(text: str, is_extended: bool) -> list[str]

grapheme_indices(text: str, is_extended: bool) -> list[tuple[int, str]]

split_word_bounds(text: str) -> list[str]

split_word_bound_indices(text: str) -> list[tuple[int, str]]

unicode_words(text: str) -> list[str]

unicode_sentences(text: str) -> list[str]

line_breaks(text: str) -> list[tuple[int, bool]]

line_break_units(text: str) -> list[str]

text_width(text: str) -> int

gettext_wrap(text: str, width: int) -> list[str]

Building for Distribution

Running Tests

License

Project links

Key dates

Owner

Credits

GitHub Statistics

Requires

Classifiers

Download files

Source Distribution

Built Distributions

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

`graphemes(text: str, is_extended: bool) -> list[str]`

`grapheme_indices(text: str, is_extended: bool) -> list[tuple[int, str]]`

`split_word_bounds(text: str) -> list[str]`

`split_word_bound_indices(text: str) -> list[tuple[int, str]]`

`unicode_words(text: str) -> list[str]`

`unicode_sentences(text: str) -> list[str]`

`line_breaks(text: str) -> list[tuple[int, bool]]`

`line_break_units(text: str) -> list[str]`

`text_width(text: str) -> int`

`gettext_wrap(text: str, width: int) -> list[str]`

Algorithm	Hash digest
SHA256	`2d827bbd6d211d6f4e18d6c5bc87694b75625a710f19e773e1caa7c9add17538`
MD5	`e63a8f252430162f222b520f6c88435b`
BLAKE2b-256	`41225bbfba1ae91ea0f96536d776a198c4fc1d0be3eec0d1d74c1a311ab11e7d`

Algorithm	Hash digest
SHA256	`9365c3d2b69ac64a851b684fee4c03ecf68319abda803cc4c1040165e0bd8eba`
MD5	`6b7367727c194c9c2a0e77540bce8a3f`
BLAKE2b-256	`7ee44a4e86bd174be870ac68c4b2c5e8d9f84e7b98c06bfc138d470123807d32`

Algorithm	Hash digest
SHA256	`847bd22046b2699c959773909e61411722e97d7d8c8bdef909d192992a5c9f18`
MD5	`81f41f9309ae853ce2351882d6899dee`
BLAKE2b-256	`f32b559e7d8951894749e834ecaec3952ce08b6e3d8505805858507d9b402df4`

Algorithm	Hash digest
SHA256	`fc610b0e110e1e767b445f33a66f294812cf473b10708e312c72e7bf33b948be`
MD5	`f57589b32d8c5db1ffd40e13eef316b2`
BLAKE2b-256	`e2334586211caa0ea38854be57fa2b3087ecd469117b9802859c13b57f47830c`

Algorithm	Hash digest
SHA256	`52c761ae41a230330c5c90e6408c243997818b4cc8cfb401e34f29969ba210e7`
MD5	`ac111c1e781997408a57343d714b429b`
BLAKE2b-256	`5969e89c452cbce965b59e42a5f66ffb03cbd3a3a2ac60160d441b09c6057a13`

Algorithm	Hash digest
SHA256	`d886ed76bf7c5fd61ce3e21b6fc837eb5325275321423af44e5ba77e35ee50ed`
MD5	`6f3350420481225754851a0f6941903f`
BLAKE2b-256	`a1c208571487b2cdc00fc2af78ffb50c86cf53ca2a34f33d0c55aba912c1c8e5`

Algorithm	Hash digest
SHA256	`5c5d84ef817b40aaa1518fdfc7c902cf4ba712db160fe3e6a2f84db0c1a4cc0a`
MD5	`0e80f74fb5eed27fa636cd0726ed17d6`
BLAKE2b-256	`d1b5d44a9eef3d43cb784fcd682b0b190b8373dee7858dcd8605e3aa184da0af`

Algorithm	Hash digest
SHA256	`b6a7747972fd27e1c2a0cf7bee1cb3138891ddcb22aa934dd83f70d752a2cdab`
MD5	`3653c7456f3cfb4d870727f348ce5fb1`
BLAKE2b-256	`86eb0c71fc29b984e38ed941703e6cac11329a2c0db0de56a451dbc540cd67c5`

Algorithm	Hash digest
SHA256	`ddbd44a0ef56abb3ceb66e22df55b0cfbdc4df2397708fdbef5d6d22f2a3e19e`
MD5	`f420b7a174d757b8d9fe285400cbcb7d`
BLAKE2b-256	`9652fa2e774955dee89aad2c2922fb88b68c96ac0c30407d2ce014581a01ae8f`

Algorithm	Hash digest
SHA256	`a34801ff2b4e49dfb0b58c42db0badd971244ac8b71921b357729a1a496d361b`
MD5	`dc0c6bb6c3c9e1aaa997a1d68be1f7c6`
BLAKE2b-256	`14208a16e751943851328ccd5ae962c3abaa9f5d0eafbfa136e71d0b3593f73a`