TextPreprocessor

Overview

The TextPreprocessor is a Python class designed for comprehensive text preprocessing. It facilitates tasks like removing links, hashtags, special characters, emojis, numbers, and stopwords. Additionally, it provides functionality for converting text to lowercase.

Installation

Ensure you have NLTK installed. You can install NLTK via pip:

pip install nltk

Usage

### Import the TextPreprocessor class
from text_preprocessor import TextPreprocessor
# Initialize the preprocessor with default settings
preprocessor = TextPreprocessor()

# Customize the preprocessor by setting flags
preprocessor = TextPreprocessor(
    remove_links=True,
    remove_hashtags=True,
    remove_characters=True,
    convert_to_lowercase=True,
    remove_emojis=True,
    remove_numbers=True,
    remove_stopwords_flag=True
)



text = "Your text goes here..."


processed_text = preprocessor.preprocess_text(text)

Available Methods

1. preprocess_text(text): Preprocesses the input text based on the initialized flags.
2. Other methods in the class can be used individually for specific preprocessing steps (e.g., remove_links, remove_stopwords, etc.).

Examples

text = "Hello! This is an example text with #hashtags and links: https://example.com"



# Initialize preprocessor
preprocessor = TextPreprocessor(remove_links=True, remove_hashtags=True)

# Preprocess text
processed_text = preprocessor.preprocess_text(text)
print(processed_text)
Output: "Hello This is an example text with and links"

License

This project is licensed under the MIT License - see the LICENSE file for details.