politenessr
Project description
Politenessr
Intro
Politenessr is a package used to predict the value of politeness of texts.
It is based on a fine tuned BERT model.
Install
Use pip
If pip
is installed, politenessr could be installed directly from it:
pip install politenessr
Dependencies
python>=3.6.0
torch>=0.4.1
numpy
pandas
unidecode
pytorch-pretrained-bert
pytorch-transformers
Usage and Example
Notes: During your first usage, the package will download a model file automatically, which is about 400MB.
predict
predict
is the core method of this package,
which takes a single text of a list of texts, and returns a list of raw values in [1,5]
(higher means more politeness, while lower means less).
Simplest usage
You may directly import politenessr
and use the default predict method, e.g.:
>>> import politenessr
>>> politenessr.predict("I am totally agree with you")
[4.3568916]
Construct from class
Alternatively, you may also construct the object from class, where you could customize the model path and device:
>>> from politenessr import Politenessr
>>> pr = Politenessr()
# Predict a single text
>>> pr.predict("I am totally agree with you")
[3.5638056]
# Predict a list of texts
>>> preds = pr.predict(['I am totally agree with you','I hate you'])
>>> f"Raw values are {preds}"
[3.5638053 2.2007465]
More detail on how to construct the object is available in docstrings.
Model using multiprocessing when preprocessing a large dataset into BERT input features
If you want to use several cpu cores via multiprocessing while preprocessing a large dataset, you may construct the object via
>>> pr = Politenessr(CPU_COUNT=cpu_cpunt, CHUNKSIZE=chunksize)
If you want to faster the code through multi gpus, you may construct the object via
>>> pr = Politenessr(is_paralleled=False, BATCH_SIZE = batch_size)
Contact
Junjie Wu (wujj38@mail2.sysu.edu.cn)
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for politenessr-1.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4fd4ed17e6dbdea760c731c359a50d54c161740bd4bd7f14ad68e4ab10fe4a7b |
|
MD5 | 30484af0d7afb0e3bcee74d32d165968 |
|
BLAKE2b-256 | 88dece5b437ac7a1be964da783fc10f69d26cca935806b376a090ab9a1fe1103 |