turnchunk

Chunking that understands who is speaking.
Parse any transcript, chunk it without ever splitting a speaker turn, and keep the speaker and timestamp on every chunk.

---

90 people on GitHub have written a function called def chunk_by_speaker. 2,080 more have hand-rolled it inline. Nobody packaged it.

chonkie ships 11 chunkers. LangChain ships 11 text splitters. LlamaIndex ships 7 node parsers. Not one of the 29 understands a conversation. So everyone building over meetings, interviews, calls, podcasts or depositions writes the same 200 lines, slightly differently, and gets the same details wrong.

Every number above is a live query. Run scripts/verify-prior-art.sh and check.

What goes wrong without it

The red chunk is Dana arguing against a discount. The splitter cut her turn in half, and the second half carries no name — so retrieved on its own, nothing says who it belongs to. A RAG answer to "who wanted to hold the price?" has nothing to go on but a guess, and attributing a commitment to the wrong person is worse than not finding it at all.

That's the whole product. Every chunk keeps its speaker and its timestamp.

$ turnchunk viz examples/demo.vtt --compare --target 230   # reproduce it

The GIF is generated by running the real CLI (scripts/make_demo_gif.py), so it cannot drift from what the tool actually prints.

Install

pip install turnchunk        # Python 3.9+
npm install turnchunk        # Node 18+, browsers, Deno, Bun, edge runtimes

Zero dependencies in both. Installs in under a second, runs in a Lambda, on a Raspberry Pi, inside an air-gapped network.

30 seconds

from turnchunk import parse, chunk

turns  = parse("meeting.vtt")          # format auto-detected from content
chunks = chunk(turns, target=2000, overlap=200)

for c in chunks:
    print(c.primary_speaker, c.start_ms, c.text[:80])

import { parse, chunk } from "turnchunk";

const turns  = parse(vttText);         // format auto-detected from content
const chunks = chunk(turns, { target: 2000, overlap: 200 });

for (const c of chunks) {
  console.log(c.primarySpeaker, c.startMs, c.text.slice(0, 80));
}

The two implementations are provably identical. Not "ported carefully" -- verified. tests/corpus/conformance.json is generated from the Python implementation and both test suites assert against it, down to the SHA-256 chunk ids:

PY : 84145dc64967320e Alice Chen    ebb887b32ecae650 Bob Ferreira
TS : 84145dc64967320e Alice Chen    ebb887b32ecae650 Bob Ferreira

CI fails if regenerating the corpus produces a diff, so the two cannot drift.

Every chunk carries what you need to cite it:

c.id                # stable, content-addressed - usable as a vector-store key
c.text              # "Dana Okafor: If we're going to give ground on price..."
c.speakers          # ["Dana Okafor"]
c.primary_speaker   # "Dana Okafor"  (overlap excluded)
c.start_ms          # 41600      -> jump a player to the moment it was said
c.end_ms            # 62300
c.turn_start        # 5          -> index back into the transcript
c.overlap_indices   # [0]        -> which turns were carried from the last chunk

Reads anything

Detected from file content, never the extension — exports are routinely saved with the wrong suffix.

Format	Handles
WebVTT	Teams <v Speaker> spans, Zoom inline names, YouTube rolling captions, per-word <00:00:01.234> timestamps, NOTE/STYLE blocks, cue settings
SubRip	numbered blocks, HTML markup, inline speakers
Plain text	[00:12] Alice: · Alice: · **Alice:** · Alice (0:12): · Otter's name-then-time layout · wrapped paragraphs
Whisper	openai-whisper, faster-whisper, WhisperX, OpenAI verbose_json, diarized or not
Deepgram	utterances, paragraphs, or word-level with speaker ids
AssemblyAI	utterances or words — milliseconds, not seconds
Rev.ai	monologues with punctuation elements
Speechmatics	results with per-word speakers
Anything else	any list of {speaker, text, start, end} objects

Three of these are quietly hostile:

• YouTube auto-captions scroll. Each cue repeats the previous cue's tail. Concatenate them naively and most of your transcript appears two or three times, silently doubling your index. turnchunk detects the overlap and reports how many cues it removed.
• AssemblyAI uses milliseconds while everyone else uses seconds. Read it wrong and the text is perfect while every citation points at the wrong moment, forever.
• A cue is not a turn. Subtitle cues break every few seconds for display. Chunking on cues instead of turns makes speaker-aware chunking pointless, so parse() merges consecutive same-speaker cues into real turns.

The four rules

1. A chunk boundary only ever falls on a speaker turn boundary. No chunk contains the tail of one person's answer glued to the start of another's.
2. A turn longer than the target splits at sentence boundaries, and every piece keeps its speaker. Splitting a monologue is unavoidable; losing the attribution is not.
3. Overlap is whole turns, and carried-over turns are marked in overlap_indices so aggregates don't count a speaker twice.
4. Short tails merge backwards. A 40-character trailing chunk is a stub; stubs match everything weakly and crowd real content out of the top-k.

chunk(turns,
      target=2000,           # measured on the rendered text, label included
      overlap=200,           # rounded up to whole turns
      min_tail_ratio=1/3,    # tails below this fraction merge backwards
      size_fn=len)           # pass a tokeniser to budget in tokens instead

Budget in tokens without bundling a 100MB tokenizer:

import tiktoken
enc = tiktoken.get_encoding("cl100k_base")
chunk(turns, target=512, size_fn=lambda s: len(enc.encode(s)))

Speakers get resolved

One transcript refers to the same person four ways. Left alone that's four "different" people, and filtering by participant silently returns a third of their turns.

from turnchunk.speakers import resolve_speakers

turns, mapping = resolve_speakers(turns)
# {"john": "John Smith", "JOHN SMITH": "John Smith", "J. Smith": "John Smith"}

$ turnchunk speakers interview.vtt

  John Smith   ←  john, JOHN SMITH, J. Smith, John (Host)
  Dana Okafor  ←  dana okafor

  2 distinct speakers from 6 labels

It refuses to guess. J. Smith alongside both John Smith and Jane Smith stays separate — merging there would misattribute what someone said, which is the one failure this library must never produce. SPEAKER_00 and SPEAKER_01 are never merged either. Override explicitly when you know:

resolve_speakers(turns, rename={"SPEAKER_00": "Alice Chen"})

Drop into your pipeline

# LangChain
from turnchunk.integrations.langchain import TurnChunkSplitter
docs = TurnChunkSplitter(chunk_size=2000).split_transcript("meeting.vtt")

# LlamaIndex
from turnchunk.integrations.llamaindex import TurnChunkNodeParser
nodes = TurnChunkNodeParser(chunk_size=2000).parse_transcript("meeting.vtt")

# chonkie-shaped
from turnchunk.integrations.chonkie import ConversationChunker
chunks = ConversationChunker(chunk_size=2000)("meeting.vtt")

Speaker, time span and turn range land in the document metadata, so you can filter by participant and jump a player to the second it was said.

Render it for a prompt

from turnchunk.render import to_context_all
prompt_block = to_context_all(chunks)      # overlap dropped automatically

[meeting.vtt 00:41–01:02]
(00:41) Dana Okafor: Three outages is right, but two of those were upstream.
(00:52) Priya Raman: That's fair. Did they put it in writing?

CLI

turnchunk viz meeting.vtt --compare     # see it against a naive splitter
turnchunk chunk meeting.vtt --json      # chunks with full metadata
turnchunk chunk meeting.vtt --context   # rendered for a prompt
turnchunk stats meeting.vtt             # speakers, talk time, duration
turnchunk speakers meeting.vtt          # resolved identity mapping
turnchunk detect transcripts/*          # what format is this, really?
turnchunk report transcripts/*.vtt --fail-on-issues   # CI gate

Fast

Measured on real YouTube caption exports, not synthetic data:

Transcript	Size	Parse	Chunk
Conference talk	18 K chars, 498 cues	5 ms	1 ms
Full course	259 K chars, 2,935 cues	14 ms	11 ms
16 MB export	1.7 M chars, 46,958 cues	450 ms	16 ms

Two quadratic blowups turned up when those real files were first run through it — one slicing the remaining document inside a loop, one rebuilding a string on every merged cue. Both are fixed, and tests/test_performance.py asserts scaling stays linear so they can't come back.

What's guaranteed

The central claim is not asserted, it's tested. test_a_turn_is_never_split generates 40 transcripts, runs each through 27 configurations, and checks every turn that fits the target:

40 generated transcripts × 27 configs = 1,080 chunk runs
21,924 individual "this turn was not split" assertions

Also covered: no content is lost; overlap is always a whole-turn suffix; short tails merge; oversized turns keep their speaker on every piece; unknown timestamps stay None and never become 0; chunk ids are deterministic across runs and machines; ambiguous speaker names never merge; and the LangChain and LlamaIndex adapters are exercised against the real frameworks in CI — which runs everything on Python 3.9–3.13 across Linux, macOS and Windows, and the TypeScript port on Node 18, 20 and 22.

Parsers are regression-tested against structures taken from genuine exports, including the case where YouTube writes <i> for italics — which used to leak a literal <i> into the chunk text, and was only ever going to be found by running a real file through it.

pip install -e ".[dev]" && pytest      # 131 Python tests
cd js && npm ci && npm test            # 75 TypeScript conformance tests

Scope

turnchunk does one thing: transcript in, well-formed chunks out.

It does not transcribe, embed, store or retrieve, and it never calls a model. That's why it has no dependencies and why it composes with whatever you already use. If you want the surrounding pipeline, bring your own — this is the piece in the middle that nobody had written.

Prior art, honestly

There is no packaged speaker-aware chunker on PyPI, npm, or GitHub — that's what scripts/verify-prior-art.sh checks. The closest things:

Project	What it is	Speaker-aware?
chonkie	The chunking library. 11 chunkers, excellent.	No
LangChain text-splitters	11 splitters by language and markup	No
LlamaIndex node parsers	7 parsers including semantic	No
semchunk	Fast semantic splitting	No

If you only need prose chunking, use chonkie — it's better at that than this will ever be. turnchunk exists for the case where who said it is part of the answer.

Contributing

The most valuable contribution is a transcript format that breaks the parser. Real exports beat synthetic fixtures every time. Drop a file (redacted as needed) in an issue, or add it to tests/fixtures/ with a test.

See CONTRIBUTING.md.

License

MIT