turnchunk

Chunking that understands who is speaking.
Parse any transcript, chunk it without ever splitting a speaker turn, and keep the speaker and timestamp on every chunk.

---

90 people on GitHub have written a function called def chunk_by_speaker. 2,080 more have hand-rolled it inline. Nobody packaged it.

chonkie ships 11 chunkers. LangChain ships 11 text splitters. LlamaIndex ships 7 node parsers. Not one of the 29 understands a conversation. So everyone building over meetings, interviews, calls, podcasts or depositions writes the same 200 lines, slightly differently, and gets the same details wrong.

Every number above is a live query. Run scripts/verify-prior-art.sh and check.

What goes wrong without it

The red chunk is Dana arguing against a discount. The splitter cut her turn in half, and the second half carries no name — so retrieved on its own, nothing says who it belongs to. A RAG answer to "who wanted to hold the price?" has nothing to go on but a guess, and attributing a commitment to the wrong person is worse than not finding it at all.

That's the whole product. Every chunk keeps its speaker and its timestamp.

$ turnchunk viz examples/demo.vtt --compare --target 230   # reproduce it

The GIF is generated by running the real CLI (scripts/make_demo_gif.py), so it cannot drift from what the tool actually prints.

Install

pip install turnchunk        # Python 3.9+
npm install turnchunk        # Node 18+, browsers, Deno, Bun, edge runtimes

Zero dependencies in both. Installs in under a second, runs in a Lambda, on a Raspberry Pi, inside an air-gapped network.

30 seconds

from turnchunk import parse_file, chunk

turns  = parse_file("meeting.vtt")     # format auto-detected from content
chunks = chunk(turns, target=2000, overlap=200)

for c in chunks:
    print(c.primary_speaker, c.start_ms, c.text[:80])

Already have the text? parse(text) takes content directly. It never touches the filesystem — a string is always transcript content, never a path, so parse(request.body) on untrusted input cannot be tricked into reading a file off your server. Use parse_file() for paths you control.

import { parse, chunk } from "turnchunk";

const turns  = parse(vttText);         // format auto-detected from content
const chunks = chunk(turns, { target: 2000, overlap: 200 });

for (const c of chunks) {
  console.log(c.primarySpeaker, c.startMs, c.text.slice(0, 80));
}

The two implementations are provably identical. Not "ported carefully" -- verified. tests/corpus/conformance.json is generated from the Python implementation and both test suites assert against it, down to the SHA-256 chunk ids:

PY : 84145dc64967320e Alice Chen    ebb887b32ecae650 Bob Ferreira
TS : 84145dc64967320e Alice Chen    ebb887b32ecae650 Bob Ferreira

CI fails if regenerating the corpus produces a diff, so the two cannot drift.

Every chunk carries what you need to cite it:

c.id                # stable, content-addressed - usable as a vector-store key
c.text              # "Dana Okafor: If we're going to give ground on price..."
c.speakers          # ["Dana Okafor"]
c.primary_speaker   # "Dana Okafor"  (overlap excluded)
c.start_ms          # 41600      -> jump a player to the moment it was said
c.end_ms            # 62300
c.turn_start        # 5          -> index back into the transcript
c.overlap_indices   # [0]        -> which turns were carried from the last chunk

Safe to point at user uploads

parse() treats a str as transcript content, always. It never opens a file, so handing it untrusted input cannot disclose anything from your disk. parse_file() is the only thing that reads from the filesystem, and a pathlib.Path also works where a file is unambiguously meant.

Speaker resolution is bounded too. Transcripts with implausibly many distinct speakers — usually a misparsed log or a hostile upload rather than a real conversation — skip the partial-name merging step rather than doing quadratic work on it. Case and punctuation folding still runs.

Reads anything

Detected from file content, never the extension — exports are routinely saved with the wrong suffix.

Format	Handles
WebVTT	Teams <v Speaker> spans, Zoom inline names, YouTube rolling captions, per-word <00:00:01.234> timestamps, NOTE/STYLE blocks, cue settings
SubRip	numbered blocks, HTML markup, inline speakers
Plain text	[00:12] Alice: · Alice: · **Alice:** · Alice (0:12): · Otter's name-then-time layout · wrapped paragraphs
Whisper	openai-whisper, faster-whisper, WhisperX, OpenAI verbose_json, diarized or not
Deepgram	utterances, paragraphs, or word-level with speaker ids
AssemblyAI	utterances or words — milliseconds, not seconds
Rev.ai	monologues with punctuation elements
Speechmatics	results with per-word speakers
AWS Transcribe	items joined to speaker_labels segments — times are strings
Google Cloud STT	word lists with "1.500s" times and speakerTag, cumulative results de-duplicated
Azure Speech	recognizedPhrases — 100-nanosecond ticks, not seconds
Anything else	any list of {speaker, text, start, end} objects

Several of these are quietly hostile:

• YouTube auto-captions scroll. Each cue repeats the previous cue's tail. Concatenate them naively and most of your transcript appears two or three times, silently doubling your index. turnchunk detects the overlap and reports how many cues it removed.
• Every vendor encodes time differently, and each is a way to be silently wrong. AssemblyAI emits integer milliseconds, AWS writes seconds as strings, Google appends an "s" suffix, and Azure counts 100-nanosecond ticks. Read any of them as plain seconds and the text is perfect while every citation points at the wrong moment, forever.
• Google's diarized results are cumulative — the final result repeats every word. Concatenate them and you get the transcript two or three times over.
• AWS keeps diarization in a separate list, addressed by time range rather than attached to the words, and its punctuation items carry no speaker at all.
• A cue is not a turn. Subtitle cues break every few seconds for display. Chunking on cues instead of turns makes speaker-aware chunking pointless, so parse() merges consecutive same-speaker cues into real turns.

The four rules

1. A chunk boundary only ever falls on a speaker turn boundary. No chunk contains the tail of one person's answer glued to the start of another's.
2. A turn longer than the target splits at sentence boundaries, and every piece keeps its speaker. Splitting a monologue is unavoidable; losing the attribution is not.
3. Overlap is whole turns, and carried-over turns are marked in overlap_indices so aggregates don't count a speaker twice.
4. Short tails merge backwards. A 40-character trailing chunk is a stub; stubs match everything weakly and crowd real content out of the top-k.

chunk(turns,
      target=2000,           # measured on the rendered text, label included
      overlap=200,           # rounded up to whole turns
      min_tail_ratio=1/3,    # tails below this fraction merge backwards
      size_fn=len)           # pass a tokeniser to budget in tokens instead

Budget in tokens without bundling a 100MB tokenizer:

import tiktoken
enc = tiktoken.get_encoding("cl100k_base")
chunk(turns, target=512, size_fn=lambda s: len(enc.encode(s)))

Speakers get resolved

One transcript refers to the same person four ways. Left alone that's four "different" people, and filtering by participant silently returns a third of their turns.

from turnchunk.speakers import resolve_speakers

turns, mapping = resolve_speakers(turns)
# {"john": "John Smith", "JOHN SMITH": "John Smith", "J. Smith": "John Smith"}

$ turnchunk speakers interview.vtt

  John Smith   ←  john, JOHN SMITH, J. Smith, John (Host)
  Dana Okafor  ←  dana okafor

  2 distinct speakers from 6 labels

It refuses to guess. J. Smith alongside both John Smith and Jane Smith stays separate — merging there would misattribute what someone said, which is the one failure this library must never produce. SPEAKER_00 and SPEAKER_01 are never merged either. Override explicitly when you know:

from turnchunk.speakers import resolve_speakers

resolve_speakers(turns, rename={"SPEAKER_00": "Alice Chen"})

Catches diarizer mistakes it can't fix

A perfectly chunked turn can still carry the wrong name, because the diarizer upstream got it wrong. turnchunk can't repair that — it would need the audio, or a model, and being model-free is the point. What it can do is notice the fingerprints a diarizer error leaves in timing and text:

$ turnchunk lint meeting.vtt

  [high]   mid_utterance_flip   turns 4-5 @01:02
           'Priya Raman' stops mid-sentence and 'Marcus Bell' starts 50ms later with a
           lowercase continuation -- likely one utterance split across two labels

  [medium] flapping             turns 12-17 @03:44
           6 consecutive turns of <= 4 words alternating between 'A' and 'B' --
           diarizers flap like this on a single voice; real back-channel has one long side

  2 finding(s)  heuristics only -- labels are never changed; raw_speaker keeps the original

Three signals, each labelled with a confidence:

Finding	What it looks like	Why a diarizer does it
mid-utterance flip	speaker changes with no pause, and the sentence continues	one utterance split in two, second half given to someone else
flapping	a run of very short turns bouncing between the same two labels	one voice the diarizer can't settle on
ghost speaker	a SPEAKER_03 with a handful of words in an hour of audio	over-segmentation noise

The false-positive traps are handled on purpose. Real back-channel — one person talking, the other saying "mhm" — is not flapping, because only one side is short. A named person who spoke once is never a ghost; that's a real person. Fast turn-taking between finished sentences is normal conversation, not a flip.

from turnchunk import diarization_warnings

for f in diarization_warnings(turns):
    print(f.confidence, f.kind, f.start_index, f.reason)

Nothing is ever changed. The labels stay exactly as the source gave them, raw_speaker preserves the original, and the caller decides. Misattribution is worse than a miss, and a "fix" that guesses wrong is misattribution with extra steps. turnchunk lint --fail-on high makes it a CI gate.

Hand the windows to something that has the audio. Every timed finding is a closed window, start_ms–end_ms, covering every turn it names. A pipeline that can re-diarize doesn't have to redo the whole file — it re-checks just the flagged regions:

$ turnchunk lint meeting.vtt --json
[{"kind": "mid_utterance_flip", "start_ms": 62000, "end_ms": 71400, ...}]

turnchunk finds the suspects cheaply; whatever owns the audio confirms them.

Drop into your pipeline

# LangChain
from turnchunk.integrations.langchain import TurnChunkSplitter
docs = TurnChunkSplitter(chunk_size=2000).split_transcript("meeting.vtt")

# LlamaIndex
from turnchunk.integrations.llamaindex import TurnChunkNodeParser
nodes = TurnChunkNodeParser(chunk_size=2000).parse_transcript("meeting.vtt")

# chonkie-shaped
from turnchunk.integrations.chonkie import ConversationChunker
chunks = ConversationChunker(chunk_size=2000)("meeting.vtt")

Speaker, time span and turn range land in the document metadata, so you can filter by participant and jump a player to the second it was said.

Render it for a prompt

from turnchunk.render import to_context_all
prompt_block = to_context_all(chunks)      # overlap dropped automatically

[meeting.vtt 00:41–01:02]
(00:41) Dana Okafor: Three outages is right, but two of those were upstream.
(00:52) Priya Raman: That's fair. Did they put it in writing?

CLI

turnchunk viz meeting.vtt --compare     # see it against a naive splitter
turnchunk chunk meeting.vtt --json      # chunks with full metadata
turnchunk chunk meeting.vtt --context   # rendered for a prompt
turnchunk stats meeting.vtt             # speakers, talk time, duration
turnchunk speakers meeting.vtt          # resolved identity mapping
turnchunk lint meeting.vtt --fail-on high   # flag likely diarizer mistakes
turnchunk detect transcripts/*          # what format is this, really?
turnchunk report transcripts/*.vtt --fail-on-issues   # CI gate

Fast

Measured on real YouTube caption exports, not synthetic data:

Transcript	Size	Parse	Chunk
Conference talk	18 K chars, 498 cues	5 ms	1 ms
Full course	259 K chars, 2,935 cues	14 ms	11 ms
16 MB export	1.7 M chars, 46,958 cues	450 ms	16 ms

Two quadratic blowups turned up when those real files were first run through it — one slicing the remaining document inside a loop, one rebuilding a string on every merged cue. Both are fixed, and tests/test_performance.py asserts scaling stays linear so they can't come back.

What's guaranteed

The central claim is not asserted, it's tested. test_a_turn_is_never_split generates 40 transcripts, runs each through 27 configurations, and checks every turn that fits the target:

40 generated transcripts × 27 configs = 1,080 chunk runs
21,924 individual "this turn was not split" assertions

Also covered: no content is lost; overlap is always a whole-turn suffix; short tails merge; oversized turns keep their speaker on every piece; unknown timestamps stay None and never become 0; chunk ids are deterministic across runs and machines; ambiguous speaker names never merge; and the LangChain and LlamaIndex adapters are exercised against the real frameworks in CI — which runs everything on Python 3.9–3.13 across Linux, macOS and Windows, and the TypeScript port on Node 18, 20 and 22.

Parsers are regression-tested against structures taken from genuine exports, including the case where YouTube writes <i> for italics — which used to leak a literal <i> into the chunk text, and was only ever going to be found by running a real file through it.

pip install -e ".[dev]" && pytest      # 178 Python tests
cd js && npm ci && npm test            # 95 TypeScript conformance tests

Scope

turnchunk does one thing: transcript in, well-formed chunks out.

It does not transcribe, embed, store or retrieve, and it never calls a model. That's why it has no dependencies and why it composes with whatever you already use. If you want the surrounding pipeline, bring your own — this is the piece in the middle that nobody had written.

Prior art, honestly

There is no packaged speaker-aware chunker on PyPI, npm, or GitHub — that's what scripts/verify-prior-art.sh checks. The closest things:

Project	What it is	Speaker-aware?
chonkie	The chunking library. 11 chunkers, excellent.	No
LangChain text-splitters	11 splitters by language and markup	No
LlamaIndex node parsers	7 parsers including semantic	No
semchunk	Fast semantic splitting	No

If you only need prose chunking, use chonkie — it's better at that than this will ever be. turnchunk exists for the case where who said it is part of the answer.

Contributing

The most valuable contribution is a transcript format that breaks the parser. Real exports beat synthetic fixtures every time. Drop a file (redacted as needed) in an issue, or add it to tests/fixtures/ with a test.

See CONTRIBUTING.md.

License

MIT