privyscope-en 0.1.2

privyscope-en — English language pack for the privyscope PII engine

privyscope-en

English PII detection & masking engine — part of the privyscope series. Detects and masks person names, phone numbers, national IDs, emails, addresses, financial info, private dates, and credentials in English text.

⚠️ privyscope is a redaction aid, not an anonymization or compliance guarantee. See Limitations.

Install

pip install privyscope-en

60-second quickstart

Python

from privyscope_en import Privyscope

engine = Privyscope.from_pretrained()                 # downloads ONNX weights on first run
result = engine.redact("John Smith's number is 555-123-4567")

result.masked_text        # "<PER>'s number is <PHONE>"
result.detected_spans     # [DetectedSpan(label="PER", start=0, end=10, ...), ...]
result.summary            # {"span_count": 2, "by_label": {"PER": 1, "PHONE": 1}, ...}

CLI

privyscope redact "John Smith's number is 555-123-4567"
cat notes.txt | privyscope redact --operating-point high_recall

Documentation

Full guides live in docs/ — organised by what you want to do:

I want to…	Guide
Run it from the terminal	CLI Reference
Call it from Python	Python API Reference
Understand the JSON output	Output Schemas
Score it on my labelled data	Evaluation & Output Modes
Trade precision vs recall	Operating Points
Run it offline / air-gapped	Offline Usage
Fine-tune on my own data	Fine-tuning

Entities

PER · PHONE · ID_NUM · EMAIL · LOC · BANK · DATE · SECRET. Regex patterns live in privyscope/regex_rules.yaml (user-extensible, no code change); EN-specific extended entities in privyscope/entity_config.yaml.

How it works

A two-stage hybrid pipeline (SRS §3.4), results merged via Union:

1. Regex filter — structurally obvious PII (phone, email, IDs, cards, secrets).
2. ONNX NER — a BIOES token classifier with a constrained Viterbi decoder for contextual PII (names, addresses, private dates).

Inference is ONNX Runtime only — no PyTorch at runtime. Recall-first, with runtime operating-point tuning (no retraining). PyTorch is needed only to fine-tune.

Model & performance

• Architecture — roberta-base encoder → BIOES token-classification head → constrained Viterbi decoder.

• Runtime artifact — INT8-quantized ONNX, ~120 MB (under the ≤ 150 MB budget), max sequence length 256. Weights download from Hugging Face Hub on first use, with a SHA-256 checksum.txt for integrity verification.

• Accuracy — entity-level strict micro-F1 = 0.997 on a held-out validation set (2,000 sentences, disjoint from training; typed/strict scoring over the full regex + NER pipeline). Per-entity strict F1:

  PER	LOC	DATE	ID_NUM	BANK	PHONE	SECRET
  1.00	1.00	1.00	0.99	0.99	0.98	1.00
  (I think it is overfit...)

  (EMAIL is matched deterministically by the regex stage; the sample contained no EMAIL instances.) The set is disjoint from training, so the score reflects generalization rather than memorization — out-of-distribution text (unusual names or contexts) will score lower. Reproduce with privyscope eval --lang en your_val.jsonl; see Evaluation & Output Modes.

Limitations

• Not an anonymization/compliance guarantee; use as one layer of privacy-by-design.
• Known failure modes: under-detection of uncommon/regional names; over-redaction of public entities in ambiguous contexts; fragmented spans in heavily mixed-format text; missed SECRET for novel credential formats.
• Extra human review recommended for medical/legal/financial/government workflows.

License

Apache-2.0. Weights are distributed on Hugging Face Hub under Apache-2.0 with a checksum.txt (SHA-256) for integrity verification. Contributions welcome — see CONTRIBUTING.md.