Skip to main content

A simple tool for Vietnamese Sentiment Analysis

Project description

A Simple Tool For Sentiment Analysis

Sentivi - a simple tool for sentiment analysis which is a wrapper of scikit-learn and PyTorch Transformers models (for more specific purpose, it is recommend to use native library instead). It is made for easy and faster pipeline to train and evaluate several classification algorithms.

Documentation: https://sentivi.readthedocs.io/en/latest/index.html

Classifiers

  • <input type="checkbox" checked="" disabled="" /> Decision Tree
  • <input type="checkbox" checked="" disabled="" /> Gaussian Naive Bayes
  • <input type="checkbox" checked="" disabled="" /> Gaussian Process
  • <input type="checkbox" checked="" disabled="" /> Nearest Centroid
  • <input type="checkbox" checked="" disabled="" /> Support Vector Machine
  • <input type="checkbox" checked="" disabled="" /> Stochastic Gradient Descent
  • <input type="checkbox" disabled="" /> Character Convolutional Neural Network
  • <input type="checkbox" checked="" disabled="" /> Multi-Layer Perceptron
  • <input type="checkbox" checked="" disabled="" /> Long Short Term Memory
  • <input type="checkbox" checked="" disabled="" /> Text Convolutional Neural Network
  • <input type="checkbox" checked="" disabled="" /> Transformer
  • <input type="checkbox" disabled="" /> Ensemble
  • <input type="checkbox" disabled="" /> Lexicon-based

Text Encoders

  • <input type="checkbox" checked="" disabled="" /> One-hot
  • <input type="checkbox" checked="" disabled="" /> Bag of Words
  • <input type="checkbox" checked="" disabled="" /> Term Frequency - Inverse Document Frequency
  • <input type="checkbox" checked="" disabled="" /> Word2Vec
  • <input type="checkbox" checked="" disabled="" /> Transformer Tokenizer (for Transformer classifier only)
  • <input type="checkbox" disabled="" /> WordPiece
  • <input type="checkbox" disabled="" /> SentencePiece

Install

  • Install legacy version from PyPI:

    pip install sentivi
    
  • Install latest version from source:

    git clone https://github.com/vndee/sentivi
    cd sentivi
    pip install .
    

Example

from sentivi import Pipeline
from sentivi.data import DataLoader, TextEncoder
from sentivi.classifier import SVMClassifier
from sentivi.text_processor import TextProcessor

text_processor = TextProcessor(methods=['word_segmentation', 'remove_punctuation', 'lower'])

pipeline = Pipeline(DataLoader(text_processor=text_processor, n_grams=3),
                    TextEncoder(encode_type='one-hot'),
                    SVMClassifier(num_labels=3))

train_results = pipeline(train='./data/dev.vi', test='./data/dev_test.vi')
print(train_results)

pipeline.save('./weights/pipeline.sentivi')
_pipeline = Pipeline.load('./weights/pipeline.sentivi')

predict_results = _pipeline.predict(['hàng ok đầu tuýp có một số không vừa ốc siết. chỉ được một số đầu thôi .cần '
                                    'nhất đầu tuýp 14 mà không có. không đạt yêu cầu của mình sử dụng',
                                    'Son đẹpppp, mùi hương vali thơm nhưng hơi nồng, chất son mịn, màu lên chuẩn, '
                                    'đẹppppp'])
print(predict_results)
print(f'Decoded results: {_pipeline.decode_polarity(predict_results)}')

Take a look at more examples in example/.

Pipeline Serving

Sentivi use FastAPI to serving pipeline. Simply run a web service as follows:

# serving.py
from sentivi import Pipeline, RESTServiceGateway

pipeline = Pipeline.load('./weights/pipeline.sentivi')
server = RESTServiceGateway(pipeline).get_server()
# pip install uvicorn python-multipart
uvicorn serving:server --host 127.0.0.1 --port 8000

Access Swagger at http://127.0.0.1:8000/docs or Redoc http://127.0.0.1:8000/redoc. For example, you can use curl to send post requests:

curl --location --request POST 'http://127.0.0.1:8000/get_sentiment/' \
     --form 'text=Son đẹpppp, mùi hương vali thơm nhưng hơi nồng'

# response
{ "polarity": 2, "label": "#POS" }

Deploy using Docker

FROM tiangolo/uvicorn-gunicorn-fastapi:python3.7

COPY . /app

ENV PYTHONPATH=/app
ENV APP_MODULE=serving:server
ENV WORKERS_PER_CORE=0.75
ENV MAX_WORKERS=6
ENV HOST=0.0.0.0
ENV PORT=80

RUN pip install -r requirements.txt
docker build -t sentivi .
docker run -d -p 8000:80 sentivi

Future Releases

  • Lexicon-based
  • CharCNN
  • Ensemble learning methods
  • Model serving (Back-end and Front-end)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for sentivi, version 1.1.0
Filename, size File type Python version Upload date Hashes
Filename, size sentivi-1.1.0.tar.gz (19.8 kB) File type Source Python version None Upload date Hashes View
Filename, size sentivi-1.1.0-py3-none-any.whl (27.2 kB) File type Wheel Python version py3 Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page