filler-prediction

thai filler word prediction for voice bots. classifies customer input into categories and returns the appropriate filler phrase to play instantly while the llm generates a full response.

built for ingfah.ai voice bot but easily adaptable to any thai voice ai system.

why

voice bots have a latency problem: the user speaks, asr transcribes, then the llm takes 1-3 seconds to respond. dead silence feels broken. the solution is to play a short filler phrase ("สักครู่นะคะ", "ขออภัยด้วยน่ะคะ") immediately while the llm thinks.

but you can't play the same filler for everything. if someone is angry, "ได้เลยค่ะ" sounds dismissive. if someone asks a question, "ขออภัยด้วยน่ะคะ" makes no sense.

this classifier picks the right filler by category.

categories

default filler phrases

a random filler is picked from the matching category each time. these are designed to be short (~0.3-0.5s when synthesized) for minimal latency.

how it works

uses intfloat/multilingual-e5-small embeddings with centroid-based cosine similarity:

1. each category has ~30-60 anchor phrases (real thai customer service examples)
2. on init, all anchors are embedded and averaged into category centroids
3. at inference, the input is embedded and compared to centroids via cosine similarity
4. the closest category wins, and a random filler from that category is returned

performance

• accuracy: 89.6% on 1,000 thai customer service sentences
• inference: <10ms per classification (after model load)
• init: ~200ms for centroid computation
• model size: ~118mb (multilingual-e5-small)

installation

pip install filler-prediction

usage

from filler_prediction import FillerClassifier

# loads model automatically on first init
clf = FillerClassifier()

# classify and get category + confidence + filler
category, confidence, filler = clf.classify("อยากถามเรื่องบิลครับ")
# ("question", 0.872, "สักครู่นะคะ")

category, confidence, filler = clf.classify("ใช้งานไม่ได้เลย")
# ("complaint", 0.891, "ขออภัยด้วยน่ะคะ")

category, confidence, filler = clf.classify("ได้ครับ ตกลง")
# ("default", 0.845, "ได้เลยค่ะ")

# or just get the filler phrase directly
filler = clf.get_filler("มีโปรอะไรบ้างครับ")
# "ตรวจสอบให้นะคะ"

sharing the model

if you alr have a SentenceTransformer instance loaded (e.g., for other tasks), pass it in to avoid loading twice:

from sentence_transformers import SentenceTransformer
from filler_prediction import FillerClassifier

model = SentenceTransformer("intfloat/multilingual-e5-small")
clf = FillerClassifier(model=model)

customizing fillers

override CATEGORY_FILLERS to use your own phrases:

import filler_prediction

filler_prediction.CATEGORY_FILLERS["complaint"] = ["ขออภัยค่ะ", "เข้าใจค่ะ"]
filler_prediction.CATEGORY_FILLERS["question"] = ["รอสักครู่นะคะ"]

license

mit