Real-Time Tweets Sentiment Analysis Package
Overview
Retrieving real-time tweets using twitter API, Apache Kafka, and Apache Spark Streaming; then, using tensorflow deep learning model to classify the tweets wether they positive, negative, or neutral; all in a pypi package.
TweetsAnalysis
The streamer and model package, available on pypi TweetsAnalysis
Package Requirements
- gensim
- pandas
- pyspark
- kafka-python
- streamlit
- scikit-learn
- seaborn
- tensorflow
- tweepy==3.9.0
- pydantic
- strictyaml
- joblib
Model
The model architecture:
The model results in about 85.5% in the train set and 84.4% accuracy on the test set, which has 160000 tweets; therefore, there is no over-fitting here.
Run
First we need to install the requirements with:
pip install TweetsAnalysis
To train the model run, but first we need to specifiy the model and data directories in the config file:
python train_model.py
Straming
Start kafka with:
bin/zookeeper-server-start.sh config/zookeeper.properties
bin/kafka-server-start.sh config/server.properties
then create a kafka topic (tweets_stream) with:
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic tweets_stream
1 maintainer
Metadata
1 maintainer
Release files for TweetsAnalysis 1.1.8
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| TweetsAnalysis-1.1.8.tar.gz | 17.5 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| TweetsAnalysis-1.1.8-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 29.8 kB
Release files / TweetsAnalysis-1.1.8.tar.gz
| Download URL | TweetsAnalysis-1.1.8.tar.gz |
|---|---|
| Size | 17.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
20a1c7b990a13e665c06353f61847369f5409a07ebcbec681e2045961253248d
|
|
BLAKE2b-256 checksum How to use checksums |
9d0ff6a5368294edc7dec5c3db38a8c044367d1773cf90062f20fe67619179e9
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.10.4
|
Release files / TweetsAnalysis-1.1.8-py3-none-any.whl
| Download URL | TweetsAnalysis-1.1.8-py3-none-any.whl |
|---|---|
| Size | 12.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
62f1fbc34d64967055fad4d9cd11ed3ad4b68e3b86952f9720d42188752f736e
|
|
BLAKE2b-256 checksum How to use checksums |
bf3a916e9f6a58a574ab9f8d061518571ba259011037aead55fd69d20d5e6269
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.10.4
|
