filler-prediction 0.5.2

thai filler word prediction for voice bots - picks the right acknowledgment phrase while llm thinks

filler-prediction

thai filler word prediction for voice bots. classifies customer input into categories and returns the appropriate filler phrase to play instantly while the llm generates a full response.

built for ingfah.ai voice bot but easily adaptable to any thai voice ai system.

why

voice bots have a latency problem: the user speaks, asr transcribes, then the llm takes 1-3 seconds to respond. dead silence feels broken. the solution is to play a short filler phrase ("สักครู่นะคะ", "ขออภัยด้วยน่ะคะ") immediately while the llm thinks.

but you can't play the same filler for everything. if someone is angry, "ได้เลยค่ะ" sounds dismissive. if someone asks a question, "ขออภัยด้วยน่ะคะ" makes no sense.

this classifier picks the right filler by category.

categories

category	when	example fillers
complaint	angry, frustrated, profanity, threats	ขออภัยด้วยน่ะคะ
question	asking for info, pricing, how-to	สักครู่นะคะ, ตรวจสอบให้นะคะ
default	greetings, agreements, requests, everything else	รับทราบค่ะ, ได้เลยค่ะ

default filler phrases

category	fillers
complaint	ขออภัยด้วยน่ะคะ
question	สักครู่นะคะ, สักครู่ค่ะ, ตรวจสอบให้นะคะ
default	รับทราบค่ะ, ค่ะ ได้ค่ะ, ได้เลยค่ะ, ดีเลยค่ะ, ยินดีค่ะ

a random filler is picked from the matching category each time. these are designed to be short (~0.3-0.5s when synthesized) for minimal latency.

how it works

uses intfloat/multilingual-e5-small embeddings with centroid-based cosine similarity:

1. each category has ~30-60 anchor phrases (real thai customer service examples)
2. on init, all anchors are embedded and averaged into category centroids
3. at inference, the input is embedded and compared to centroids via cosine similarity
4. the closest category wins, and a random filler from that category is returned

performance

• accuracy: 89.6% on 1,000 thai customer service sentences
• inference: <10ms per classification (after model load)
• init: ~200ms for centroid computation
• model size: ~118mb (multilingual-e5-small)

installation

pip install filler-prediction

usage

from filler_prediction import FillerClassifier

# loads model automatically on first init
clf = FillerClassifier()

# classify and get category + confidence + filler
category, confidence, filler = clf.classify("อยากถามเรื่องบิลครับ")
# ("question", 0.872, "สักครู่นะคะ")

category, confidence, filler = clf.classify("ใช้งานไม่ได้เลย")
# ("complaint", 0.891, "ขออภัยด้วยน่ะคะ")

category, confidence, filler = clf.classify("ได้ครับ ตกลง")
# ("default", 0.845, "ได้เลยค่ะ")

# or just get the filler phrase directly
filler = clf.get_filler("มีโปรอะไรบ้างครับ")
# "ตรวจสอบให้นะคะ"

sharing the model

if you alr have a SentenceTransformer instance loaded (e.g., for other tasks), pass it in to avoid loading twice:

from sentence_transformers import SentenceTransformer
from filler_prediction import FillerClassifier

model = SentenceTransformer("intfloat/multilingual-e5-small")
clf = FillerClassifier(model=model)

customizing fillers

override CATEGORY_FILLERS to use your own phrases:

import filler_prediction

filler_prediction.CATEGORY_FILLERS["complaint"] = ["ขออภัยค่ะ", "เข้าใจค่ะ"]
filler_prediction.CATEGORY_FILLERS["question"] = ["รอสักครู่นะคะ"]

license

mit