Text Preprocessing Python Package

Course Link: Introduction to NLP

This Python package is created by Uditya Narayan Tiwari. It provides various text preprocessing utilities for natural language processing (NLP) tasks.

Installation from PyPi

You can install this package using pip as follows:

pip install nlp_text_preprocessing

Installation from GitHub

You can install this package from GitHub as follows:

pip install git+https://github.com/udityamerit/Text-Processing-Package-For-Natural-Language-Processing.git --upgrade --force-reinstall

Uninstall the Package

To uninstall the package, use the following command:

pip uninstall nlp_text_preprocessing

Requirements

You need to install these python packages.

python -m spacy download en_core_web_sm
spacy
textblob
beautifulsoup4
nltk
openpyxl
SpeechRecognition==3.10.4
pyaudio==0.2.14
PrettyTable
scikit-learn
wordcloud
lxml
pandas
numpy
matplotlib

How to Use the Package

1. Basic Text Preprocessing

Lowercasing Text

import nlp_text_preprocessing as tp

text = "HELLO WORLD!"
processed_text = tp.to_lower_case(text)
print(processed_text)  # Output: hello world!

Expanding Contractions

import nlp_text_preprocessing as tp

text = "I'm learning NLP."
processed_text = tp.contraction_to_expansion(text)
print(processed_text)  # Output: I am learning NLP.

Removing Emails

import nlp_text_preprocessing as tp

text = "Contact me at example@example.com"
processed_text = tp.remove_emails(text)
print(processed_text)  # Output: Contact me at 

Removing URLs

import nlp_text_preprocessing as tp

text = "Check out https://example.com"
processed_text = tp.remove_urls(text)
print(processed_text)  # Output: Check out

Removing HTML Tags

import nlp_text_preprocessing as tp

text = "<p>Hello World!</p>"
processed_text = tp.remove_html_tags(text)
print(processed_text)  # Output: Hello World!

Removing Special Characters

import nlp_text_preprocessing as tp

text = "Hello @World! #NLP"
processed_text = tp.remove_special_chars(text)
print(processed_text)  # Output: Hello World NLP

2. Advanced Text Processing

Lemmatization

import nlp_text_preprocessing as tp

text = "running runs"
processed_text = tp.lemmatize(text)
print(processed_text)  # Output: run run

Sentiment Analysis

import nlp_text_preprocessing as tp

text = "I love programming!"
sentiment = tp.sentiment_analysis(text)
print(sentiment)  # Output: Sentiment(polarity=0.5, subjectivity=0.6)

Detecting and Translating Language

import nlp_text_preprocessing as tp
from googletrans import Translator

translator = Translator()
text = "Bonjour tout le monde"
lang = tp.detect_language(text, translator)
translated_text = tp.translate(text, 'en', translator)
print(f"Language: {lang}, Translated: {translated_text}")
# Output: Language: fr, Translated: Hello everyone

3. Feature Extraction

Word Count

import nlp_text_preprocessing as tp

text = "I love NLP."
count = tp.word_count(text)
print(count)  # Output: 3

Character Count

import nlp_text_preprocessing as tp

text = "I love NLP."
count = tp.char_count(text)
print(count)  # Output: 9

N-Grams

import nlp_text_preprocessing as tp

text = "I love NLP"
ngrams = tp.n_grams(text, n=2)
print(ngrams)  # Output: [('I', 'love'), ('love', 'NLP')]

4. Full Example: Cleaning Text

Here’s an example of how you might use several functions together to clean text data:

import nlp_text_preprocessing as tp

text = "I'm loving this NLP tutorial! Contact me at https://www.linkedin.com/in/uditya-narayan-tiwari-562332289/  Visit https://udityanarayantiwari.netlify.app/"
cleaned_text = tp.clean_text(text)
print(cleaned_text)
# Output: i am loving this nlp tutorial contact me at visit

One Short Feature Extraction

import nlp_text_preprocessing as tp

tp.extract_features("I love NLP")

Notes

• Be cautious when using heavy operations like lemmatize and spelling_correction on very large datasets, as they can be time-consuming.
• The package supports custom cleaning and preprocessing pipelines by using these modular functions together.