privacy-steward

A CLI tool that redacts PII from plain-text files using the openai/privacy-filter model via a native PyTorch implementation. All inference runs locally — no data ever leaves your machine, making it a natural fit for GDPR-regulated environments where personal data must not be transferred to external processors (Articles 25 and 44).

Why privacy-steward?

The official OpenAI opf CLI processes one file at a time, requires manual installation, and offers limited placeholder control. privacy-steward is a drop-in alternative built for practitioners who need to sanitise datasets at scale:

Installation

No installation required — run directly with uvx (part of uv):

uvx privacy-steward notes.txt

Or install permanently to add privacy-steward to your PATH:

uv tool install -U privacy-steward

First run downloads the openai/privacy-filter model weights and caches them in ~/.cache/huggingface/hub/. Subsequent runs are fully offline.

Requirements: Python 3.12 or later.

Quick start

# Redact a single .txt file (output: notes.redacted.txt alongside the source)
privacy-steward notes.txt

# Write a single-file result into an existing output directory
privacy-steward notes.txt --output ./clean/

# Redact an entire directory of .txt files, write to a custom output location
privacy-steward ./corpus/ --output ./corpus_clean/

# Show detected entities as they are processed (-v)
privacy-steward notes.txt -v

# Preview without writing files
privacy-steward ./corpus/ --dry-run

# Write an aggregate JSON summary report
privacy-steward ./corpus/ --output ./corpus_clean/ --report

Default output format

By default each detected entity is replaced with a typed label that reflects what was found:

Hi, my name is <PRIVATE_PERSON> and I work at Acme Corp.
You can reach me at <PRIVATE_EMAIL> or call me at <PRIVATE_PHONE>.
My home address is <PRIVATE_ADDRESS>.
The meeting is on <PRIVATE_DATE>. Visit us at <PRIVATE_URL>.
Please send the invoice to account number <ACCOUNT_NUMBER>.
My API key is <SECRET>.

Pass --placeholder to override: any literal string, or use {entity_type} for interpolation (e.g. --placeholder "[{entity_type}]" → [PRIVATE_PERSON]).

Output layout

For a single-file input, the input must be a .txt file. For a directory input, redacted .txt files mirror the source tree, non-.txt files are skipped, and an .audit/ directory is always created alongside the redacted outputs:

corpus_clean/
├── chapter1.redacted.txt
├── chapter2.redacted.txt
├── subdir/
│   └── chapter3.redacted.txt
└── .audit/
    ├── chapter1.audit.json    ← offsets, labels, and scores for auditing
    ├── chapter2.audit.json
    └── subdir/
        └── chapter3.audit.json

Each audit JSON records the source path, destination path, and every detected span (character offsets, entity type, and confidence score). To avoid re-exposing the PII that was just redacted, audit records omit the original matched text by default. Pass --include-text-in-audit only when you intentionally need surface forms in the audit trail and can protect the .audit/ directory accordingly.

Options

Benchmark vs. OpenAI privacy-filter CLI (opf)

Both tools process text files through the OpenAI privacy-filter model family on CPU. opf uses a custom bundled runtime; privacy-steward uses a native PyTorch implementation loaded directly from the model's safetensors weights.

Hardware: Apple MacBook Pro (2020), Apple M1, 8-core CPU (4 Performance + 4 Efficiency), 16 GB unified memory. No GPU acceleration — all inference on CPU.

Setup: 10 synthetic files across diverse document types (emails, chat logs, support tickets, contracts, invoices, etc.), 573–1,362 tokens per file, single process.

Benchmarks are reproducible: uv run python benchmarks/benchmark_throughput.py (requires benchmarks/data/ to be present).

Development

uv sync                                  # install all deps
make lint                                # ruff + mypy
make test                                # fast unit tests only
pytest -m slow                           # integration tests (require model)
uv run python benchmarks/benchmark_throughput.py  # run benchmarks