stream2sentence
Turn a stream of LLM tokens into complete sentences as soon as each one is finished, so text-to-speech can start talking right away.
Why you need this
A voice assistant receives its answer from an LLM a few tokens at a time. Each simple way of handing that text to text-to-speech (TTS) has a problem:
| Approach | Result |
|---|---|
| Wait for the full answer | Long silence before the assistant says anything |
| Send each word to TTS | Choppy speech without natural intonation |
Split on every . ? ! |
Breaks on Dr., 3.14, U.S., v2.0 or e.g. and speaks half-sentences |
| stream2sentence | Hands TTS each sentence as soon as it is really finished |
The hard part: when a . arrives, you cannot yet tell whether the sentence is over or whether it is Dr., 3. (followed by 14) or U.S. (followed by -only). stream2sentence reads a few characters past each candidate period, decides, and then releases the sentence:
Stream so far Decision
"Dr." wait: could be a title
"Dr. Smith measured 3." wait: could be a decimal number
"Dr. Smith measured 3.5 mm. T" release "Dr. Smith measured 3.5 mm."
With the recommended English setup, the sentence above is released two characters after its period: the space and the first letter of the next sentence.
Demo
https://github.com/user-attachments/assets/0428d806-4b30-47fa-9cfe-8fb42c95b2e4
How to read the video: the text is streamed in one character at a time, the finest-grained version of an LLM token stream. When stream2sentence confirms a sentence, the line snaps into a finished sentence. The few characters that move down to the next line show how far it had to read past the period to be sure.
Every sentence in the demo is packed with traps for naive splitters: titles and initials (Dr. A. J. Rivera, Mr. O'Neil), decimals and versions (3.14159, v2.0.0-beta.3), URLs and e-mail addresses, IP addresses (10.0.0.5:443), legal citations (Smith v. Jones, cf. Brown), list markers (a. setup, b. train), and brand names with punctuation (Yahoo! Finance, Guess?). All 12 sentences come out exactly as expected, each released two characters after its real end.
Run it yourself (needs the nltk extra): python tests/show_consensus_stress_split.py
Quick start: LLM to TTS
pip install "stream2sentence[nltk]"
from stream2sentence import generate_sentences
for sentence in generate_sentences(
llm_token_stream, # any iterator of text chunks
tokenizer="nltk+rule-based", # recommended for English
auto_context=True, # release safe boundaries early
quick_yield_single_sentence_fragment=True, # start speaking even sooner
):
tts.speak(sentence) # your TTS engine
Want the whole pipeline ready-made? RealtimeTTS uses stream2sentence exactly this way: feed it an LLM stream and it speaks each sentence as soon as stream2sentence releases it.
Table of Contents
- Features
- Installation
- Usage
- Which settings do I want?
- Configuration
- Time based strategy
- Contributing
- License
Features
- Releases each sentence a few characters after it ends, so speech can start while the LLM is still writing.
- Avoids false splits on abbreviations, initials, decimals, versions, URLs, citations and list markers.
- Can release the first fragment of a reply early (for example at a comma) for faster first speech.
- Accepts chunks of any size: single characters, tokens or longer pieces.
- Synchronous, asynchronous and push-style (
SentenceSplitter) APIs. - Built-in rule-based tokenizer with optional NLTK and Stanza support.
- Optional cleanup of links and emojis before text reaches TTS.
Installation
Requires Python 3.10 or newer.
pip install stream2sentence
The base installation uses stream2sentence's built-in rule-based tokenizer and does not install NLTK or Stanza. Install either tokenizer only when you need it:
pip install "stream2sentence[nltk]"
pip install "stream2sentence[stanza]"
pip install "stream2sentence[nltk,stanza]" # both, using standard pip extras
pip install "stream2sentence[all]" # both
Usage
Pass a generator of characters or text chunks to generate_sentences() to get a generator of sentences in return.
Here's a basic example:
from stream2sentence import generate_sentences
# Dummy generator for demonstration
def dummy_generator():
yield "This is a sentence. And here's another! Yet, "
yield "there's more. This ends now."
for sentence in generate_sentences(dummy_generator()):
print(sentence)
This will output:
This is a sentence.
And here's another!
Yet, there's more.
This ends now.
For speech synthesis, enable quick_yield_single_sentence_fragment to start speaking before the first full sentence is available. See tests/test_stream_from_llm.py for a live API example.
Use generate_sentences_async() with an asynchronous input iterator. For a push-style integration, create a SentenceSplitter, call add(chunk), consume stream() after each addition, and consume flush() at the end. It drains pending input and resets the splitter for reuse; another flush without new input yields nothing. Exhaust each returned iterator before adding more input.
Recommended English setup
For English streams where the optional NLTK extra is installed, the recommended configuration is tokenizer="nltk+rule-based" with auto_context=True. This combines NLTK sentence splitting with stream2sentence's local boundary checks and allows safe sentence boundaries to be yielded earlier than the fixed context window when both checks support the split.
for sentence in generate_sentences(
text_stream,
tokenizer="nltk+rule-based",
language="en",
auto_context=True,
):
print(sentence)
Keep already-queued sentences together
Start speaking early, then keep already-queued sentences together.
When the complete text is already queued, AvailableTextRun keeps a later
We can do that tomorrow, though. together instead of sending though.
to speech synthesis on its own. The first fragment keeps its usual early release.
A later sentence may wait for the usual sentence-end confirmation instead of
being released at the comma. Pass only text you already have; do not wait to
fill a batch. Ordinary strings keep the existing behavior.
See the example and integration guide.
Which settings do I want?
- No extra dependencies: the defaults (built-in rule-based tokenizer).
- Best English accuracy and lowest delay:
tokenizer="nltk+rule-based"withauto_context=True(needs thenltkextra). - Speech should start sooner: the
quick_yield_*options under Yield Control. - Output paced to speaking speed: the time based strategy.
Configuration
The generate_sentences() function offers various parameters to fine-tune its behavior:
Core Parameters
-
generator: Iterable[str | AvailableTextRun]- The primary input source, yielding chunks of text to be processed.
- Can be any iterator that emits text chunks of any size.
- AvailableTextRun marks text already queued. It keeps later complete sentences together; their output still follows normal sentence-end checks.
-
context_size: int = 12- Number of characters considered for sentence boundary detection.
- Larger values improve accuracy but may increase latency.
- Default: 12 characters
-
context_size_look_overhead: int = 12- Additional characters to examine beyond
context_sizefor sentence splitting. - Enhances sentence detection accuracy.
- Default: 12 characters
- Additional characters to examine beyond
-
auto_context: bool = False- Yields safe sentence boundaries before the full
context_sizedelay when the boundary heuristic and tokenizer agree. - Falls back to the normal
context_sizeandcontext_size_look_overheadbehavior when the boundary is still ambiguous. - Recommended for English when used with
tokenizer="nltk+rule-based". - Default: False
- Yields safe sentence boundaries before the full
-
minimum_sentence_length: int = 10- Minimum character count for a text chunk to be considered a sentence.
- Shorter fragments are buffered until this threshold is met.
- Default: 10 characters
-
minimum_first_fragment_length: int = 10- Minimum character count required for the first sentence fragment.
- Ensures the initial output meets a specified length threshold.
- Default: 10 characters
Yield Control
These parameters control how quickly and frequently the generator yields sentence fragments:
-
quick_yield_single_sentence_fragment: bool = False- When True, yields the first fragment of the first sentence as quickly as possible.
- Useful for getting immediate output in real-time applications like speech synthesis.
- Default: False
-
quick_yield_for_all_sentences: bool = False- When True, yields the first fragment of every sentence as quickly as possible.
- Extends the quick yield behavior to all sentences, not just the first one.
- Automatically sets
quick_yield_single_sentence_fragmentto True. - Default: False
-
quick_yield_every_fragment: bool = False- When True, yields every fragment of every sentence as quickly as possible.
- Provides the most granular output, yielding fragments as soon as they're detected.
- Automatically sets both
quick_yield_for_all_sentencesandquick_yield_single_sentence_fragmentto True. - Default: False
Text Cleanup
-
cleanup_text_links: bool = False- When True, removes hyperlinks from the output sentences.
- Default: False
-
cleanup_text_emojis: bool = False- When True, removes emoji characters from the output sentences.
- Default: False
-
filter_first_non_alnum_characters: bool = False- When True, removes leading non-alphanumeric characters from every emitted sentence or sentence fragment, including quick yields and the final tail.
- Filtering runs after sentence splitting and the other text cleanup options. Outputs containing no alphanumeric characters are omitted; punctuation inside an output is preserved.
- Intended for TTS systems that require an alphanumeric first character. This is destructive: leading
-3becomes3,$5becomes5, and opening quotes are removed. Normalize meaningful prefixes first, for example-3tominus 3. - Default: False (preserves leading punctuation).
Tokenization
-
tokenize_sentences: Callable = None- Custom function accepting text and returning an iterable of sentence strings, including any unfinished tail. Lists and one-shot iterators are supported.
- Return unchanged substrings to preserve their original separators. Rewritten output uses a single space between pieces.
- If None, uses
tokenizer.
-
tokenizer: str = "rule-based"- Specifies the tokenizer to use. Options: "nltk", "stanza", "rule-based", or "nltk+rule-based"
- "nltk+rule-based" splits at boundaries accepted by both NLTK and the rule-based heuristic tokenizer, plus rule-based boundaries promoted by high-confidence local checks.
- Recommended for English, especially with
auto_context=True. - Default: "rule-based"
-
language: str = "en"- Language setting for the tokenizer.
- Selects rule-based language data and the Stanza pipeline. For Stanza, use a supported language code such as
"en"or"zh";"multilingual"is not a sentence-tokenization pipeline. - NLTK uses its English sentence model, including in
"nltk+rule-based". NLTK and Stanza download model data on first use. - Default: "en"
-
never_split_numbers: bool = False- Treats bare integers followed by a period, such as
1., as non-terminal in rule-based boundary checks. Useful for numbered text; it can also hold back a sentence ending in an integer.
- Treats bare integers followed by a period, such as
Debugging and Fine-tuning
-
debug: bool = False- Prints internal boundary and yield decisions.
-
log_characters: bool = False- When True, logs each processed character to the console.
- Useful for debugging or monitoring real-time processing.
- Default: False
-
sentence_fragment_delimiters: str = ".?!;:,\n…)]}。!?؟।۔-"- Characters considered as potential sentence fragment delimiters.
- Used for quick yielding of sentence fragments.
- Default: ".?!;:,\n…)]}。!?؟।۔-"
-
full_sentence_delimiters: str = ".?!\n…。!?؟।۔"- Characters considered as full sentence delimiters.
- Used for more definitive sentence boundary detection.
- Default: ".?!\n…。!?؟।۔"
-
force_first_fragment_after_words: int = 30- Forces the yield of the first sentence fragment after this many words.
- Ensures timely output even with long opening sentences.
- Default: 30 words; use
float("inf")to disable forced word-count yielding.
-
fragment_lookahead_words: int = 0- Buffers quick-yield delimiter candidates until this many complete words have been observed.
- At the threshold, a confirmed delimiter from
full_sentence_delimitersis preferred over an earlier soft delimiter such as a comma; otherwise the latest soft delimiter is used. - If the input stream ends before the threshold, the remaining text is flushed normally.
- This counts observed words, not words in the yielded fragment. For example, after eight observed words the splitter may yield an earlier two-word clause.
- Default: 0, preserving immediate quick-yield behavior.
Time based strategy
Instead of a purely lexical strategy, a time based strategy is available. A target tokens per second (tps) is input, and generate_sentences will yield the best available output (full sentence, longest fragment, or any available buffer, in that order) if it is approaching a "deadline" where what has been output would be slower than the input tps target. If LLM is more than two full sentences ahead of the target it will output a sentence even if it's ahead of the "deadline"
from stream2sentence.stream2sentence_time_based import generate_sentences_time_based
Parameters
generator (Iterator[str])- A generator that yields text chunks.
target_tps: float = 4- the rate in tokens per second you want to use to calculate deadlines for output.
- Default is 4. (approximately the speed of human speech)
lead_time: float = 1- amount of time in seconds to wait for the buffer to build for before returning values.
max_wait_for_fragments = [3, 2]- Max amount of time in seconds that the Nth sentence will wait beyond the "deadline" for a "fragment" (text preceding a fragment delimiter), which is preferred over a piece of buffer.
- The last value in the array is used for all subsequent checks.
min_output_lengths: int[] = [2, 3, 3, 4]- An array that corresponds to the minimum output size in words for the corresponding output sentence, the last value in the array is used for all remaining output.
- For example [4,5,6] would mean the first piece of output must have 4 words, the second 5 words, and all subsequent 6.
preferred_sentence_fragment_delimiters: str[] = ['. ', '? ', '! ', '\n']- Array of strings that delimit a sentence fragment. "Preferred" are checked first and always used if the fragment meets the length requirement over the other fragment delimiters.
- Note the trailing spaces, added to differentiate between values like $3.5 and a proper sentence end
sentence_fragment_delimiters: str[] = ['; ', ': ', ', ', '* ', '**', '– ']- Array of strings that are checked after "preferred" delimiters
delimiter_ignore_prefixes: str[]- Array of strings that will not be considered "delimiters" if preceded by a delimiter.
- Used to ignore common abbreviations for things like Mr. Dr. and Mrs. where we don't want to split
- Default is a long list documented in delimiter_ignore_prefixes
wait_for_if_non_fragment: str[]- Array of strings that the algorithm will not use as the last value if the whole buffer is being output (not a fragment or sentence).
- Avoids awkward pauses on common words that are unnatural to pause at.
- Default is a long list of common words documented in avoid_pause_words.py
deadline_offsets_static: float[] = [1]- Constant amount of time in seconds to subtract from the deadline for first n sentences.
- Last value applied to all subsequent sentences
deadline_offsets_dynamic: float[] = [0]:- Added to account for the time it takes a TTS engine to generate output.
- For example, if it takes your TTS engine around 1 second to generate 10 words, you can use a value of 0.1 so that the TTS generation time is included in the deadline.
- Applied to first n sentences, last value applied to all subsequent
Contributing
Any Contributions you make are welcome and greatly appreciated.
- Fork the Project.
- Create your Feature Branch (
git checkout -b feature/AmazingFeature). - Commit your Changes (
git commit -m 'Add some AmazingFeature'). - Push to the Branch (
git push origin feature/AmazingFeature). - Open a Pull Request.
Before a release, run the release checks.
License
This project is licensed under the MIT License. For more details, see the LICENSE file.
Project created and maintained by Kolja Beigel.
Metadata
Release files for stream2sentence 1.1.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| stream2sentence-1.1.2.tar.gz | 57.1 kB | Without provenance | Details for stream2sentence-1.1.2.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| stream2sentence-1.1.2-py3-none-any.whl | Python 3 | none | any | Details for stream2sentence-1.1.2-py3-none-any.whl |
Total release size: 106.4 kB
Release files / stream2sentence-1.1.2.tar.gz
| Download URL | stream2sentence-1.1.2.tar.gz |
|---|---|
| Size | 57.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
ac92e660e79b8926811d4630f88d2ee651f41c020ffa3db5aff6c34d8871e1af
|
|
BLAKE2b-256 checksum How to use checksums |
db6421ecfa72c34cc825a43157d70ee3fc6afb6ac01c46870a331f2ae1eed20e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.11.9
|
Release files / stream2sentence-1.1.2-py3-none-any.whl
| Download URL | stream2sentence-1.1.2-py3-none-any.whl |
|---|---|
| Size | 49.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
ff4a5246a0b0ee80a32d3ad0a0ed2f352682586c90347894437919df8d460c36
|
|
BLAKE2b-256 checksum How to use checksums |
3729f48b3b62f648c7cec7f3069cfe911e2089ac205b3d73ac709f9ced4aaaf2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.11.9
|
Security for stream2sentence 1.1.2
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.