mask-llm 0.1.1

Privacy-safe LLM data cleaning for fine-tuning datasets

mask-llm

Privacy-safe data cleaning for LLM fine-tuning datasets.

Your data never leaves your machine unprotected. Names, emails, and phone numbers are masked locally before being sent to any LLM API. Results are automatically restored after cleaning.

The Problem

When preparing fine-tuning datasets, your data often contains sensitive information: customer names, emails, internal project codenames. Sending this raw data to OpenAI or any cloud LLM is a privacy risk and may violate GDPR, HIPAA, or your company's data policy.

How It Works

Your Data → [Local PII Masking] → Anonymized Text → [Cloud LLM Cleaning] → [Local Restore] → Clean Data

1. PII detected and replaced locally (John Smith → [PERSON_1])
2. Only anonymized text is sent to the cleaning API
3. Cleaned result is restored with original values locally
4. Output as standard JSONL ready for fine-tuning

Install

pip install mask-llm
python -m spacy download en_core_web_sm

Quick Start

Prepare your input as a JSONL file (one JSON object per line):

{"text": "John Smith at [e-mail] needs this dataset cleaned."}
{"text": "Call Sarah on 123-456-7890 to confirm the order."}

Run the cleaner:

python -m mask_llm \
  --input your_data.jsonl \
  --output cleaned_data.jsonl \
  --key YOUR_API_KEY

Get your free API key at: https://nexus.xiaoku88.com

Output Fine-Tuning Format

Add --finetune to output standard instruction/input/output format:

python -m mask_llm \
  --input your_data.jsonl \
  --output cleaned_data.jsonl \
  --key YOUR_API_KEY \
  --finetune

Output:

{"instruction": "Clean and format the following text for fine-tuning.", "input": "John Smith at [e-mail]...", "output": "John Smith at [e-mail]..."}

Options

Option	Default	Description
--input	required	Input JSONL file
--output	required	Output JSONL file
--key	required	Your API key
--model	deepseek-chat	Model to use
--concurrency	10	Parallel requests
--finetune	false	Output fine-tuning format
--prompt	built-in	Custom system prompt

What Gets Masked

Type	Example	Masked As
Person names	John Smith	[PERSON_1]
Organizations	Apple Inc	[ORG_1]
Email addresses	[e-mail]	[EMAIL_1]
Phone numbers	123-456-7890	[PHONE_1]
IP addresses	192.168.1.1	[IP_1]
SSN	123-45-6789	[SSN_1]

License

MIT