Banglish Stopwords 🇧🇩

Banglish Stopwords is a lightweight, high-performance Python library designed to filter out stopwords from Banglish text (Bengali written in Latin/English script). It includes a dataset of 900+ Bengali stopwords and their common chatting/typing variations.

✨ Features

• 900+ Core Words: Covers almost all common Bengali stopwords and their romanized spelling variants.
• Lazy Typing Support: Automatically handles repeated characters (e.g., naaaa -> na, hbeee -> hbe).
• Punctuation Handling: Smartly cleans text while keeping punctuation intact where necessary.
• Custom Stopwords: Add your own domain-specific stopwords, or exclude default ones you want to keep.
• Layout Preserving: remove_stopwords() keeps your original spacing and line breaks intact.
• Fast Lookup: Uses optimized Python sets for O(1) performance.

🚀 Installation

You can install the library directly from PyPI using pip:

pip install banglish-stopwords

📖 Usage

Basic filtering

from banglish_stopwords import BanglishStopwords

bn = BanglishStopwords()

# Check if a single word is a stopword
bn.is_stopword("ami")      # True
bn.is_stopword("laptop")   # False

# Filter stopwords out of a full sentence
text = "Ami r tmi ekhon bhalo achhiiiii, kintu hbeee naaa! apni abar ashen."
bn.remove_stopwords(text)
# -> "bhalo achhiiiii,      ashen."
# (note: gaps where stopwords were removed are preserved, not collapsed,
# so your original spacing/line breaks stay intact for the surviving words)

Custom stopwords

Add extra words, or exclude default ones, right when you create the object:

bn = BanglishStopwords(
    extra_stopwords=["laptop", "mobile"],   # treat these as stopwords too
    exclude_stopwords=["apni"],             # keep "apni", don't filter it out
)

Or change the stopword set at runtime:

bn = BanglishStopwords()

bn.add_stopwords(["laptop"])
bn.is_stopword("laptop")   # True

bn.remove_from_stopwords(["laptop"])
bn.is_stopword("laptop")   # False

Inspecting the full stopword set

bn.get_stopwords()   # returns the complete set[str] currently in use

🗂️ Project Structure

banglish_stopwords/
├── __init__.py     # public exports
├── core.py         # matching logic (BanglishStopwords class)
└── data.py         # the stopword dataset (STOPWORDS set)
tests/
└── test_core.py    # pytest test suite

🧪 Running Tests

pip install -e .
pip install pytest
pytest tests/ -v

🤝 Contributing

Found a missing stopword or a spelling variant that should be included? Open an issue or a pull request — the dataset lives in banglish_stopwords/data.py and is a plain, sorted Python set, so it's easy to review and extend.

📄 License

MIT