NLP Text Preprocessing Utility Package

Table of Contents

• Package Overview
• Key Features
• Architecture Diagram
• Quickstart and Setup
• Complete API Reference
  • 1. General Feature Extraction
  • 2. Text Cleaning and Normalization
  • 3. Linguistic and Morphological Processing
  • 4. Visualization Suite
• Code Examples
  • Basic Text Cleaning
  • Pandas DataFrame Integration
  • Batch Feature Extraction
  • WordCloud Generation
• Troubleshooting and Resource Setup
• License and Author

Package Overview

nlp_text_preprocessing is a production-grade, modular Python library designed to streamline text cleaning, feature extraction, linguistic analysis, and visualization for Natural Language Processing (NLP) pipelines and Machine Learning workflows.

It provides 34 single-purpose functions with built-in defensive guards for None, NaN, and missing values, making it 100% crash-proof when running over Pandas DataFrames or large raw text datasets.

Key Features

• Crash-Proof Pipeline: Automatic None/NaN input sanitation across all functions.
• Lazy and Fast Model Caching: Efficient lazy-loading for spaCy (en_core_web_sm) and cached NaiveBayes sentiment analyzers.
• Comprehensive Cleaning: Strip emails, URLs, HTML, retweets, special symbols, and expand contractions.
• Feature Extraction: Extract word counts, character counts, uppercase counts, numerics, hashtags, mentions, and full feature matrices in one call.
• Linguistic Utilities: Perform lemmatization, noun phrase extraction, spelling correction, n-grams, singularization, and pluralization.
• Flexible Visualizations: Generate and save custom WordClouds to disk or display interactively.

Architecture Diagram

The package integrates spaCy, NLTK, TextBlob, BeautifulSoup4, and WordCloud through a unified API surface:

Quickstart and Setup

1. Installation

pip install nlp-text-preprocessing

2. Downloading Required Corpora and Models

To automatically download all required NLTK corpora (stopwords, movie_reviews, brown) and spaCy models (en_core_web_sm), run:

import nlp_text_preprocessing as tp

# One-time automated setup for NLTK and spaCy resources
tp.download_nltk_packages()

Complete API Reference

Below is the complete reference guide for all 34 functions available in the library:

1. General Feature Extraction

2. Text Cleaning and Normalization

3. Linguistic and Morphological Processing

4. Visualization Suite

Code Examples

1. Basic Text Cleaning

import nlp_text_preprocessing as tp

raw_text = "Check out https://example.com! Contact support@company.org or RT @user 'I'm loving #NLP'."
cleaned = tp.clean_text(raw_text)

print(cleaned)
# Output: check out contact support company org or i am love nlp

2. Pandas DataFrame Integration

All functions seamlessly support Pandas .apply() and handle missing values (NaN/None):

import pandas as pd
import nlp_text_preprocessing as tp

df = pd.DataFrame({
    'text': [
        "I'm loving this NLP package! #awesome",
        "Contact me at info@test.com or visit https://test.com",
        None
    ]
})

# Apply end-to-end text cleaning
df['clean_text'] = df['text'].apply(tp.clean_text)

# Extract word counts safely
df['word_count'] = df['text'].apply(tp.word_count)

3. Batch Feature Extraction

Extract full feature matrices directly into Pandas columns:

import pandas as pd
import nlp_text_preprocessing as tp

df = pd.DataFrame({'text': ["Hello #world! Visit https://example.com", "Contact user@test.com"]})

# Extract all features into a new DataFrame
features_df = df['text'].apply(tp.extract_features).apply(pd.Series)
print(features_df)

4. WordCloud Generation

import nlp_text_preprocessing as tp

text = "python natural language processing machine learning text mining data science spaCy nltk textblob"

# Save directly to disk without opening interactive GUI window
tp.get_wordcloud(text, save_path="wordcloud.png", show=False)

Troubleshooting and Resource Setup

If any NLTK corpus or spaCy model throws a missing resource error during runtime, simply run:

import nlp_text_preprocessing as tp
tp.download_nltk_packages()

Or manually download spaCy model via terminal:

python -m spacy download en_core_web_sm

License and Author

• Author: Uditya Narayan Tiwari
• License: MIT License (LICENSE)