mask-llm

Privacy-safe data cleaning for LLM fine-tuning datasets.

Your data never leaves your machine unprotected. Names, emails, and phone numbers are masked locally before being sent to any LLM API. Results are automatically restored after cleaning.

The Problem

When preparing fine-tuning datasets, your data often contains sensitive information: customer names, emails, internal project codenames. Sending this raw data to OpenAI or any cloud LLM is a privacy risk and may violate GDPR, HIPAA, or your company's data policy.

How It Works

Your Data → [Local PII Masking] → Anonymized Text → [Cloud LLM Cleaning] → [Local Restore] → Clean Data

1. PII detected and replaced locally (John Smith → [PERSON_1])
2. Only anonymized text is sent to the cleaning API
3. Cleaned result is restored with original values locally
4. Output as standard JSONL ready for fine-tuning

Install

pip install mask-llm
python -m spacy download en_core_web_sm

Quick Start

Prepare your input as a JSONL file (one JSON object per line):

{"text": "John Smith at john@company.com needs this dataset cleaned."}
{"text": "Call Sarah on 123-456-7890 to confirm the order."}

Run the cleaner:

python -m mask_llm \
  --input your_data.jsonl \
  --output cleaned_data.jsonl \
  --key YOUR_API_KEY

Get your free API key at: https://nexus.xiaoku88.com

Output Fine-Tuning Format

Add --finetune to output standard instruction/input/output format:

python -m mask_llm \
  --input your_data.jsonl \
  --output cleaned_data.jsonl \
  --key YOUR_API_KEY \
  --finetune

Output:

{"instruction": "Clean and format the following text for fine-tuning.", "input": "John Smith at john@company.com...", "output": "John Smith at john@company.com..."}

Options

What Gets Masked

License

MIT