isplit-rs 0.3.0

Lazy, typed string splitting helpers for Python, backed by Rust.

isplit scans a string once and yields pieces as soon as separators are found. It is useful for tokenizer-style code where callers want to consume tokens lazily instead of allocating the whole split result up front.

The iterator implementation is backed by a small PyO3 Rust extension when available, with the original pure-Python implementation kept as a fallback.

Installation

python -m pip install isplit-rs

The PyPI distribution is named isplit-rs, while the Python package is imported as isplit.

Quick start

from isplit import irsplit, isplit

list(isplit("A+B+C+D", "+"))
# ["A", "B", "C", "D"]

list(isplit("AAA,,BBB", ",,"))
# ["AAA", "BBB"]

list(irsplit("A+B+C+D", "+"))
# ["D", "C", "B", "A"]

Or consume pieces lazily:

from isplit import isplit

for token in isplit("AAA,BBB,CCC", ","):
    print(token)

Why isplit?

Performance

isplit is designed for lazy, partial consumption. When you need every piece as a list, CPython's built-in str.split is usually the best choice. When you only need the first few pieces from a larger string, isplit can stop before splitting the rest of the input.

One local CPython 3.12 / macOS arm64 run, extracting only the first field with the first separator at index 8:

In other words, isplit helps most when it avoids unnecessary work. For small strings, the difference is tiny; for larger strings, avoiding a full split can matter.

Run the benchmark locally:

uv run python scripts/benchmark_split.py

API

isplit(text: str, sep: str) -> Iterator[str]
irsplit(text: str, sep: str) -> Iterator[str]

• isplit(text, sep) yields pieces from left to right.
• irsplit(text, sep) yields pieces from right to left.

Both functions follow str.split semantics for non-empty separators and raise ValueError when sep is empty.

Verify Rust acceleration

Installed wheels include the native Rust extension. To check that your environment is using it:

python - <<'PY'
import isplit._rust as rust

print(rust.__file__)
print(list(rust.isplit("A+B+C", "+")))
PY

The printed path should point to a compiled extension such as isplit/_rust.abi3.so on Linux/macOS or isplit/_rust.pyd on Windows.