yasbd-union

Sentence splitting for when you genuinely have no idea what language you're looking at.

What this is

yasbd-union is an experimental add-on for yasbd-lib.

It takes sentence-splitting rules from every built-in language in yasbd-lib and throws them into one shared space.

Sometimes it behaves nicely. Sometimes it makes bold assumptions. Sometimes it surprises even you.

That's basically the whole deal.

auto vs xx

auto tries to be smart about it.

It looks at your text, decides what language it is, and uses the right rules for the job. Clean and structured.

xx doesn't bother with that step.

It assumes your text is already a mix of everything and just applies all available rules at once.

Install

pip install yasbd-union

Then register it:

from yasbd import BoundaryDetector

detector = BoundaryDetector(lang="xx", external_lang_packs = ["yasbd_union"])

Example

sentences = list(detector.segment(
    "Dr. Wang said 你好世界。Prof. Li replied 是的。"
))

print(sentences)

Output:

["Dr. Wang said 你好世界。", "Prof. Li replied 是的。"]

When to use xx

Use it when:

• You don't know what language your text is in
• Your input is messy, mixed, or unpredictable
• You're dealing with logs, chats, or scraped text
• You just want something that "tries its best"

When not to use xx

Avoid it when:

• You need strict, repeatable results
• Your text is single-language
• You don't want surprises in sentence boundaries
• You're trying to explain results to someone very literal

In those cases, auto or a specific language pack will behave better.

A few honest notes

• Some sentence splits will be slightly unexpected
• Results depend on what language rules are bundled in yasbd-lib
• It is not fully predictable by design

If that sounds like a problem, xx is probably not what you want.

License

MIT: If it breaks, it's still yours.