textprep-lite 0.1.0

Lightweight text cleaning and anonymization for real-world NLP

textprep-lite

Lightweight text cleaning and anonymization for real-world NLP.

textprep-lite is a tiny Python library that standardizes text normalization and anonymization in 1–2 function calls, designed for ML/NLP workflows.

Installation

pip install textprep-lite

Quickstart

from textprep_lite import clean_text, anonymize_text, preprocess_text

raw = "Hey!!   <br> This is  John   \nContact: [e-mail], [phone]"

# Basic cleaning
cleaned = clean_text(raw)
print(cleaned)
# Output: "hey!! this is john contact: [e-mail], [phone]"

# Anonymization
anon = anonymize_text(raw)
print(anon)
# Output: "Hey!!   <br> This is  John   \nContact: [REDACTED], [REDACTED]"

# Full pipeline
processed = preprocess_text(raw)
print(processed)
# Output: "hey!! this is john contact: [redacted], [redacted]"

Configuration

clean_text

Argument	Default	Description
lowercase	True	Convert text to lowercase.
strip_html	True	Remove HTML tags.
normalize_whitespace	True	Collapse multiple spaces/newlines.
strip_accents	False	Normalize Unicode accents to ASCII.
remove_punct	False	Remove punctuation.
remove_digits	False	Remove digits.
keep_newlines	False	Preserve newlines during whitespace normalization.

anonymize_text

Argument	Default	Description
mask_emails	True	Mask email addresses.
mask_phones	True	Mask phone numbers.
mask_urls	True	Mask URLs.
mask_ip	False	Mask IPv4 addresses.
replacement	"[REDACTED]"	Replacement string for masked items.

Why textprep-lite?

Real-world text is messy. You repeatedly need to normalize whitespace, strip HTML, and mask PII. Instead of copying the same regex snippets into every notebook, textprep-lite offers a tested, reusable, and composable way to clean your text data.

Testing & Quality

textprep-lite is designed for reliability.

• Unit Tests: Comprehensive coverage for all cleaning and anonymization logic.
• Stress Tested: Verified against edge cases (empty strings, emojis, mixed IP/phone formats) and synthetic logs.
• Clean API: Type-hinted and documented.

Contributing

1. Clone the repo.
2. Install dependencies (e.g., pip install -e .[dev]).
3. Run tests:

   pytest