NLP Text Preprocessing Utility Package

NLP Text Preprocessing Banner

📌 Overview

nlp_text_preprocessing is a production-grade, fast, and modular Python library designed to streamline text cleaning, feature extraction, linguistic analysis, and visualization for Natural Language Processing (NLP) pipelines and Machine Learning workflows.

Maintained by Uditya Narayan Tiwari.

🏗 Architecture Diagram

The package is built with a decoupled, high-performance architecture integrating spaCy, NLTK, TextBlob, BeautifulSoup4, and WordCloud with safe null/NaN defensive wrappers for seamless Pandas DataFrame processing.

NLP Text Preprocessing Architecture Diagram

Pipeline Architecture Workflow

flowchart TD
    Raw[Raw Input Text / Pandas Series] --> Guard[_clean_input Guard: None/NaN Safety]
    
    Guard --> Cleaning[Cleaning & Normalization Module]
    Guard --> Features[Feature Extraction Engine]
    Guard --> Linguistic[Linguistic & NLP Processing]
    Guard --> Viz[Visualization Suite]
    
    Cleaning --> Lower[to_lower_case]
    Cleaning --> Expand[contraction_to_expansion]
    Cleaning --> Email[remove_emails / count_emails]
    Cleaning --> URL[remove_urls / count_urls]
    Cleaning --> HTML[remove_html_tag]
    Cleaning --> Regex[remove_special_chars / remove_mentions / remove_rt]
    Cleaning --> Stops[remove_stop_words - Case Insensitive]
    
    Features --> Stats[word_count / char_count / avg_word_len]
    Features --> Counts[hashtags_count / mentions_count / numerics_count / upper_case_count]
    Features --> Summary[extract_features - Full Feature Dict]
    
    Linguistic --> SpaCy[spaCy Lemmatizer / convert_to_base]
    Linguistic --> NLTKCorpus[Auto-cached NLTK Corpora Check]
    Linguistic --> Sentiment[TextBlob Sentiment - NaiveBayes Cached]
    Linguistic --> Grams[n_grams / get_noun_phrase / singularize / pluralize]
    
    Viz --> Cloud[get_wordcloud - File Save & GUI options]
    
    Cleaning --> Pipeline[clean_text - End-to-End Pipeline]
    Features --> Output[ML Ready Feature Matrix / Clean Text]
    Linguistic --> Output
    Pipeline --> Output

🚀 Quick Installation

From PyPI

pip install nlp_text_preprocessing

From GitHub (Latest Development Version)

pip install git+https://github.com/udityamerit/Text-Processing-Package-For-Natural-Language-Processing.git --upgrade --force-reinstall

Automatic Resource Setup

Download required NLTK corpora (stopwords, movie_reviews, brown, etc.) and spaCy language models (en_core_web_sm):

import nlp_text_preprocessing as tp

# Downloads all required NLTK corpora automatically
tp.download_nltk_packages()

📚 Complete API Reference & Function List

Below is the complete list of all 34 functions available in the nlp_text_preprocessing package:

1. General Feature Extraction Functions

Function Name	Parameters	Return Type	Description
`word_count(x)`	`x: str`	`int`	Returns total number of whitespace-separated words.
`char_count(x)`	`x: str`	`int`	Returns total characters excluding whitespace.
`avg_word_len(x)`	`x: str`	`float`	Returns average word length (safely returns `0.0` on empty input).
`stop_words_count(x)`	`x: str`	`int`	Returns count of stop words (case-insensitive check).
`hashtags_count(x)`	`x: str`	`int`	Returns count of hashtags (`#tag`).
`mentions_count(x)`	`x: str`	`int`	Returns count of user mentions (`@user`).
`numerics_count(x)`	`x: str`	`int`	Returns count of standalone numeric tokens.
`upper_case_count(x)`	`x: str`	`int`	Returns count of uppercase words.
`extract_features(x)`	`x: str`	`dict`	Returns a dictionary containing all extracted text features at once.

2. Text Cleaning & Normalization Functions

Function Name	Parameters	Return Type	Description
`to_lower_case(x)`	`x: str`	`str`	Converts input text to lowercase.
`contraction_to_expansion(x)`	`x: str`	`str`	Expands abbreviations and contractions using contractions lookup table.
`remove_emails(x)`	`x: str`	`str`	Strips email addresses from text.
`count_emails(x)`	`x: str`	`int`	Returns count of email addresses found in text.
`remove_urls(x)`	`x: str`	`str`	Strips HTTP/HTTPS URLs and `www` links from text.
`count_urls(x)`	`x: str`	`int`	Returns count of URLs found in text.
`remove_rt(x)`	`x: str`	`str`	Removes retweet tags (`RT @user`).
`count_rt(x)`	`x: str`	`int`	Returns count of retweet markers in text.
`remove_html_tag(x)`	`x: str`	`str`	Strips HTML/XML tags using BeautifulSoup parser.
`remove_accented_chars(x)`	`x: str`	`str`	Normalizes and converts accented characters (NFKD -> ASCII).
`remove_mentions(x)`	`x: str`	`str`	Strips user handles (`@username`).
`remove_special_chars(x)`	`x: str`	`str`	Removes punctuation and non-alphanumeric special symbols.
`remove_repeated_chars(x)`	`x: str`	`str`	Truncates character repetitions beyond 2 consecutive occurrences.
`remove_stop_words(x)`	`x: str`	`str`	Removes stop words (case-insensitive, preserving capitalized text structure).
`clean_text(text)`	`text: str`	`str`	Executes full end-to-end cleaning pipeline (lowercasing, contractions, emails, URLs, HTML, special chars, lemmatization).

3. Linguistic & Morphological Processing Functions

Function Name	Parameters	Return Type	Description
`convert_to_base(x)`	`x: str`	`str`	Lemmatizes nouns and verbs via spaCy while keeping other POS tags intact.
`lemmatize(x)`	`x: str`	`str`	Performs full lemmatization across all tokens via spaCy (`en_core_web_sm`).
`correct_spelling(x)`	`x: str`	`str`	Corrects word spelling using TextBlob dictionary model.
`get_noun_phrase(x)`	`x: str`	`list[str]`	Extracts noun phrases from text.
`n_grams(x, n=2)`	`x: str, n: int`	`list`	Generates word-level n-grams.
`singularize_words(x)`	`x: str`	`str`	Converts plural nouns (`NNS`) into singular forms.
`pluralize_words(x)`	`x: str`	`str`	Converts singular nouns (`NN`) into plural forms.
`sentiment_analysis(x)`	`x: str`	`str`	Returns sentiment classification (`'pos'` / `'neg'`) using module-cached NaiveBayesAnalyzer.

4. Visualization Functions

Function Name	Parameters	Return Type	Description
`get_wordcloud(x, save_path=None, show=True)`	`x: str, save_path: str, show: bool`	`WordCloud`	Generates a WordCloud image object, with optional file saving and GUI display options.

💻 Usage Code Examples

1. Cleaning Raw Text

import nlp_text_preprocessing as tp

raw_text = "Check out https://example.com! Contact support@company.org or RT @user 'I'm loving #NLP'."
cleaned = tp.clean_text(raw_text)

print(cleaned)
# Output: check out contact support company org or i am love nlp

2. Processing Pandas DataFrames

All functions safely handle None, NaN, and missing values:

import pandas as pd
import nlp_text_preprocessing as tp

df = pd.DataFrame({
    'text': [
        "I'm loving this NLP package! #awesome",
        "Contact me at info@test.com or visit https://test.com",
        None
    ]
})

# Apply full cleaning pipeline
df['clean_text'] = df['text'].apply(tp.clean_text)

# Extract word counts
df['word_count'] = df['text'].apply(tp.word_count)

# Extract full feature dictionary as DataFrame columns
features_df = df['text'].apply(tp.extract_features).apply(pd.Series)
print(features_df)

3. WordCloud Generation & Saving

import nlp_text_preprocessing as tp

text = "python natural language processing machine learning text mining data science spaCy nltk textblob"

# Save directly to file without requiring GUI display window
tp.get_wordcloud(text, save_path="wordcloud.png", show=False)

🛠 Setup & Requirements

spacy >= 3.0
textblob
beautifulsoup4
nltk
wordcloud
pandas
numpy
matplotlib

📄 License

Distributed under the MIT License. See LICENSE for more information.

License
- OSI Approved :: MIT License
Operating System
- OS Independent
Programming Language
- Python :: 3

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

nlp_text_preprocessing-0.1.1.tar.gz (1.5 MB view details)

Uploaded Jul 25, 2026 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

nlp_text_preprocessing-0.1.1-py3-none-any.whl (12.5 kB view details)

Uploaded Jul 25, 2026 Python 3

File details

Details for the file nlp_text_preprocessing-0.1.1.tar.gz.

File metadata

Download URL: nlp_text_preprocessing-0.1.1.tar.gz
Upload date: Jul 25, 2026
Size: 1.5 MB
Tags: Source
Uploaded using Trusted Publishing? No
Uploaded via: twine/6.2.0 CPython/3.11.4

File hashes

Hashes for nlp_text_preprocessing-0.1.1.tar.gz
Algorithm	Hash digest
SHA256	`0ad7565694a4144b6a30884810406cb100351754f16ac98f0815aaed59dd1b1c`
MD5	`022ca159c94552e25a9d87d6deef217b`
BLAKE2b-256	`359f97e9b834322866642a6a21098cc2414a995282db1e0363627f870dce6ba6`

See more details on using hashes here.

File details

Details for the file nlp_text_preprocessing-0.1.1-py3-none-any.whl.

File metadata

Download URL: nlp_text_preprocessing-0.1.1-py3-none-any.whl
Upload date: Jul 25, 2026
Size: 12.5 kB
Tags: Python 3
Uploaded using Trusted Publishing? No
Uploaded via: twine/6.2.0 CPython/3.11.4

File hashes

Hashes for nlp_text_preprocessing-0.1.1-py3-none-any.whl
Algorithm	Hash digest
SHA256	`613534f1e4cbec40476d1d8b3e763b1aa1df4bfd95e28c9aafcb658b73294209`
MD5	`026eb17f5be85cf4b58d65ff4bbc48b3`
BLAKE2b-256	`20f9a3f8bf88e17194f76921f9c96098917349c56aafd699fa249bbead052446`