pypi-download-stats

Please use colab for getting no problem. For transformer model, please install simpleTransformer first or use bn_nlp for static models. I uploaded dataset and training details in my github. There is a problem in sentiment analyzer. I Will fix it soon.

SBNLTK

SUST-Bangla Natural Language toolkit. A python module for Bangla NLP tasks.
Demo Version : 1.0
NEED python 3.6+ vesrion!! Use virtual Environment for not getting unessessary Issues!!

INSTALLATION

PYPI INSTALLATION

pip3 install sbnltk
pip3 install simpletransformers

MANUAL INSTALLATION FROM GITHUB

• Clone this project
• Install all the requirements
• Call the setup.py from terminal

What will you get here?

• Bangla Text Preprocessor
• Bangla word dust,punctuation,stop word removal
• Bangla word sorting according to Bangla or English alphabet
• Bangla word normalization
• Bangla word stemmer
• Bangla Sentiment analysis(logisticRegression,LinearSVC,Multilnomial_naive_bayes,Random_Forst)
• Bangla Sentiment analysis with Bert
• Bangla sentence pos tagger (static, sklearn)
• Bangla sentence pos tagger with BERT(Multilingual-cased,Multilingual uncased)
• Bangla sentence NER(Static,sklearn)
• Bangla sentence NER with BERT(Bert-Cased, Multilingual Cased/Uncased)
• Bangla word word2vec(gensim,glove,fasttext)
• Bangla sentence embedding(Contexual,Transformer/Bert)
• Bangla Document Summarization(Feature based, Contexual, sementic Based)
• Bangla Bi-lingual project(Bangla to english google translator without blocking IP)
• Bangla document information Extraction

SEE THE CODE DOCS FOR USES!

TASKS, MODELS, ACCURACY, DATASET AND DOCS

Next releases after testing this demo

Package Installation

You have to install these packages manually, if you get any module error.

• simpletransformers
• fasttext

Models

Everything is automated here. when you call a model for the first time, it will be downloaded automatically.

With GPU or Without GPU

• With GPU, you can run any models without getting any warnings.
• Without GPU, You will get some warnings. But this will not affect in result.

Motivation

With approximately 228 million native speakers and another 37 million as second language speakers,Bengali is the fifth most-spoken native language and the seventh most spoken language by total number of speakers in the world. But still it is a low resource language. Why?

Dataset

For all sbnltk dataset and existing Dataset, see this link Bangla NLP Dataset

Trainer

For training, You can see this Colab Trainer . In future i will make a Trainer module!

When will full version come?

Very soon. We are working on paper and improvement our modules. It will be released sequentially.

About accuracy

Accuracy can be varied for the different datasets. We measure our model with random datasets but small scale. As human resources for this project are not so large.

Contribute Here

• If you found any issue, please create an issue or contact with me.