pretwita 0.1.1

A text PREprocessor for TWeets in the ITAlian language

PreTwITA

PreTwITA is an open source Preprocessor for Tweets in the ITAlian language written in Python. The purpose of such library is to provide the user with language-specific tools for text cleaning (i.e. the process of preparing raw text for Natural Language Processing).

Included features

• correction of most common italian abbreviations (e.g. xk replaced with perché)
• remove urls
• remove emojis
• remove emoticons
• remove mentions
• remove hashtags
• remove twitter reserved words (i.e. 'rt' and 'fav')
• remove stopwords
  • an option to define additional stopwords
• remove punctuation
• remove numbers
  • an option to avoid removing dates in yyyy format
• remove multiple spaces
• tokenization

Usage

For usage and tips, please refer to the demo.ipynb file