boundary-chunker

Boundary-aware semantic chunking for RAG pipelines. Wraps a ModernBERT model fine-tuned to answer one question per sentence: does a topic boundary follow this one?

Model weights: 0xKitkat/semantic-chunker-modernbert-base

Install

pip install "boundary-chunker[onnx]"   # CPU, no torch (~40MB of deps)
pip install "boundary-chunker[torch]"  # GPU

torch is not a hard dependency on purpose — a chunker belongs inside someone else's ingestion pipeline, and "requires CUDA" is the usual reason a component gets skipped.

Use

from boundary_chunker import SemanticChunker

chunker = SemanticChunker(backend="onnx")   # weights download on first use
for c in chunker.split(document):
    print(c.n_tokens, c.boundary_score, c.text)

From the command line:

boundary-chunk --backend onnx --file report.txt
cat report.txt | boundary-chunk --backend onnx --json

Or through transformers, with nothing installed from here:

from transformers import pipeline

chunk = pipeline("semantic-chunking",
                 model="0xKitkat/semantic-chunker-modernbert-base",
                 trust_remote_code=True)
chunk(document)

Knobs that matter

Does this actually help?

Sometimes. It wins clearly on boundary detection (0.82 vs 0.50 F1 against the closest existing model), but on retrieval it is roughly a wash against plain fixed-size splitting, and it loses on corpora whose documents are already shorter than the embedder's window. Splitting a 150-token document in half makes retrieval worse no matter how correct the boundary is.

Use it for long documents whose topic genuinely shifts — transcripts, reports, manuals. Measure on your own corpus before adopting it; the model card carries the full per-corpus numbers, including the losses.

Apache-2.0.