bangla-punctuation

Python inference library for restoring punctuation in unpunctuated Bengali text and automatic speech recognition (ASR) transcripts.

The package provides two checkpoints:

The extended model adds hyphen (-) and Bengali visarga (ঃ). Its rare-class scores remain uneven, so it is not necessarily better than v2 for every input.

Installation

pip install bangla-punctuation

For an NVIDIA GPU, install the PyTorch build appropriate for your CUDA setup as described at https://pytorch.org/get-started/locally/.

Python usage

from bangla_punctuation import BanglaPunctuator

punctuator = BanglaPunctuator(model="base", device="auto")
result = punctuator.restore("আজ আকাশ খুব সুন্দর তুমি কি বাইরে যাবে")
print(result)

Select the experimental model:

punctuator = BanglaPunctuator(model="extended")

Process multiple ASR results without reloading the model:

raw_transcripts = [
    "আজ আকাশ খুব সুন্দর",
    "তুমি কি বাইরে যাবে",
]
restored = punctuator.restore_batch(raw_transcripts)

Get metadata alongside the restored text:

details = punctuator.restore_with_details("আমি এখন ঢাকায় যাব তুমি যাবে")
print(details.text)
print(details.counts)
print(details.device)

A local model directory or another compatible Hub repository can also be used:

punctuator = BanglaPunctuator(model="./bangla-bert-punctuation-v2")

Command line

bangla-punctuate --model base "আজ আকাশ খুব সুন্দর তুমি কি বাইরে যাবে"

Or pipe ASR text through standard input:

printf '%s' 'আজ আকাশ খুব সুন্দর' | bangla-punctuate --model extended

Models and training data

Both models use the ELECTRA-based csebuetnlp/banglabert encoder. They are not derived from sagorsarker/bangla-bert-base.

The stable v2 model was fine-tuned for eight token-level classes using the project's original Bengali punctuation corpus. The published held-out result was approximately 0.570 punctuation macro F1 and 0.623 overall macro F1.

The extended v3 model starts from v2 and uses:

• abdullahalmunem/ha-pr-bn-munem-generated (Apache-2.0);
• BanglaPRCorpus from the BLP/EMNLP 2023 punctuation-restoration work;
• replay examples from the original training data to reduce catastrophic forgetting.

Independent macro-F1 results reported after extended training were 0.447 on Munem, 0.396 on BanglaPRCorpus, and 0.473 on the original test set. Refer to the individual Hugging Face model cards for per-class results and limitations.

ASR integration

Use punctuation restoration as a post-processing stage:

raw_text = asr.transcribe(audio)["text"]
final_text = punctuator.restore(raw_text)

For long-running services, create one BanglaPunctuator instance at startup and reuse it. Constructing it downloads/loads a checkpoint and is expensive.

Important limitations

• The input parser currently keeps Bengali-block tokens and normalizes spacing; mixed English/Bengali text is not preserved exactly.
• Punctuation is predicted after each input word. The model does not rewrite or spell-correct ASR text.
• Rare punctuation such as exclamation, semicolon, ellipsis, and the extended labels is substantially less accurate than common labels.
• Validate on transcripts from the intended ASR engine and acoustic domain.

Licensing

The Python wrapper code is MIT licensed. Model checkpoints and datasets retain their own terms. The upstream csebuetnlp/banglabert release does not currently state an explicit license, so the model repositories are marked license: other. Confirm upstream permission before commercial redistribution or use.