quick_processor

Overview

quick_processor is a Python-based text preprocessing tool designed to simplify and standardize text cleaning tasks for Natural Language Processing (NLP) applications. It leverages a variety of libraries, including NLTK, BeautifulSoup, and contractions, to provide a comprehensive suite of text cleaning functions.

Features

• Lowercase Conversion: Converts all characters in a sentence to lowercase.
• Email Removal: Removes email addresses from the text.
• Diacritic Removal: Strips diacritics from characters.
• HTML Cleaning: Removes HTML tags from the text.
• Repeated Character Replacement: Replaces repeated punctuation marks with a single occurrence.
• Emoji Translation: Translates emojis into their textual representation.
• Contraction Expansion: Expands common contractions (e.g., "can't" to "cannot").
• URL Removal: Strips URLs from the text.
• Possessive Removal: Removes possessive forms from words.
• Extra Space Removal: Eliminates extra spaces.
• Spelling Correction: Corrects spelling errors.
• Tokenization: Splits text into tokens.
• Stopword Removal: Removes common stopwords.
• Lemmatization: Reduces words to their base or root form.
• Emoticon Removal: Removes emoticons from the text.
• Non-Alphabetic Character Removal: Strips non-alphabetic characters from the text.

Usage

Here's an example of how to use quick_processor for text preprocessing:

from quick_processor import Preprocessor

# Initialize the preprocessor
preprocessor = Preprocessor()

# Sample sentence
sentence = "This is a sample sentence with an email@example.com and a link http://example.com 😊"

# Clean the sentence using default steps
cleaned_sentence = preprocessor.clean(sentence)

print(cleaned_sentence)

Contributing

Contributions are welcome! If you find any issues or have suggestions for improvements, please open an issue or submit a pull request.

With quick_processor, you can streamline your text preprocessing tasks, making your NLP pipeline more efficient and effective.