ai-data-scrubber 0.1.0

A lightweight tool for removing personal data from text before uploading to LLMs

AI Data Scrubber

The AI Data Scrubber is a lightweight privacy-focused tool designed to remove personal information from text documents before uploading them to Large Language Models (LLMs). You can use it to clean sensitive documents like resumes or contracts. It's not guaranteed to remove everything, so you should still check before you upload your file to a LLM.

It uses a mixture of regular expressions and named entity recognition models from spaCy. It's less accurate than asking a LLM to remove PII - but then you don't need to either run a LLM on your own machine, or upload your document to a LLM.

What It Does

Removes personal information like:

• Names (via AI recognition)
• Email addresses
• Phone numbers
• Street addresses & ZIP codes
• URLs
• License plates

Currently supports US formats only.

Quick Start

# Install
pip install ai-data-scrubber

# Download required language model (560MB)
python -m spacy download en_core_web_lg

# Clean your file
ai-data-scrubber your-file.txt

Usage

Command Line:

# Auto-generates output file with _scrubbed suffix
ai-data-scrubber input.txt

# Or specify output file
ai-data-scrubber input.txt -o output.txt

Python:

from ai_data_scrubber import scrub_text, scrub_file

# Scrub text directly
cleaned = scrub_text("John Smith lives at 123 Main St")

# Or scrub a file
scrub_file("input.txt", "output.txt")

Example

Before:

John Smith
123 Main Street, Apt 4B
New York, NY 10001
Email: [e-mail]
Phone: [phone]

After:

[NAME]
[ADDRESS], [UNIT]
New York, NY [ZIP_CODE]
Email: [EMAIL]
Phone: [PHONE]

Documentation

• Installation Guide - Detailed installation including from source
• Performance Testing - Test with 20 sample resumes

Requirements

• Python 3.10 or higher
• spaCy >= 3.5.0
• spaCy model: en_core_web_lg

License

This project is licensed under the MIT License - see the LICENSE file for details.

Contributing

Contributions welcome! Feel free to submit a Pull Request.