piilint
Local-first PII scanner for the files developers actually commit and send — notebooks, CSV, JSON, Parquet, and source code. Nothing leaves your machine.
Disclaimer: piilint helps you find sensitive data before it leaks. It is a detection aid, not a compliance certification, and cannot guarantee that all sensitive data is found. It does not make anyone GDPR/HIPAA/PCI compliant.
Package: piilint (PyPI name piiscan was already taken) · License: Apache-2.0 · Repo: thelonewander3r/PIIScanner
Not a secrets scanner. Pair with gitleaks or trufflehog for API keys and tokens.
Five-minute path
1. Install
Once published to PyPI (after v0.1.0 — not yet released):
# recommended for CLI use
pipx install piilint
# or one-off
uvx piilint --version
# or classic
pip install piilint
Until then, install from git or a local checkout (package is not on PyPI yet):
pipx install git+https://github.com/thelonewander3r/PIIScanner.git
# or
uv sync --extra dev
uv run piilint --version
2. Scan
piilint . # scan the current directory
piilint . --fail-on high # fail CI/pre-commit on high-severity findings
Exit codes: 0 clean / nothing staged · 1 findings at or above --fail-on · 2 usage/config/git error.
3. Wire into git / CI (optional)
Pre-commit — add to .pre-commit-config.yaml:
repos:
- repo: https://github.com/thelonewander3r/PIIScanner
rev: v0.1.0 # pin to a release tag when available
hooks:
- id: piilint
# Default: --staged --fail-on medium
GitHub Action + SARIF — drop into a workflow:
- uses: actions/checkout@v4
- name: Run piilint
id: piilint
uses: thelonewander3r/PIIScanner@main # pin to a tag when available
with:
path: .
fail-on: high
format: sarif
- name: Upload SARIF
if: always()
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: ${{ steps.piilint.outputs.sarif-path }}
4. Adopt without boiling the ocean
piilint baseline . -o piilint-baseline.json # fingerprints only — never raw PII
piilint . --baseline piilint-baseline.json # report NEW findings only
piilint . --staged # scan only git-staged files
Demo
The classic leak: a notebook runs df.head() and the output cell still holds customer rows when you commit the .ipynb.
Synthetic demo (no real PII): tests/corpus/notebook/leak_demo.ipynb
piilint tests/corpus/notebook
See also examples/README.md for a short pointer and expected story.
All of tests/corpus/ is 100% synthetic labeled data generated for tests.
Configuration
Precedence (highest wins): CLI flags → piilint.toml at the scan root → [tool.piilint] in pyproject.toml → built-in defaults.
# piilint.toml
[scan]
fail_on = "high"
min_confidence = 0.6
exclude = ["tests/fixtures/**"]
[entities]
ip_address = false
[entities.email]
severity = "medium"
[allowlist]
values = ["support@mycompany.com"]
domains = ["example.com", "mycompany.dev"]
.piiignore— gitignore-syntax path excludes (combined with.gitignore).- Inline suppressions (text/code lines):
# piilint: ignoreor# piilint: ignore[EMAIL](comma-list). Not applied to tabular/column-aggregated findings in v0. - Allowlists — exact normalized values and email domains drop matching findings.
- Test-data downweight — obvious fixtures (example.com, 555-01xx, 4111…, RFC5737 IPs) get −0.4 confidence and severity capped at low, then
min_confidenceis re-applied.
Baseline + staged
Adopt without fixing history first, and scan only what is about to land in git.
piilint baseline . -o piilint-baseline.json
piilint . --baseline piilint-baseline.json
piilint . --staged
Fingerprint design: SHA-256(relative path, entity, normalized-value hash, occurrence index). Line numbers are excluded so ordinary edits do not resurrect old findings.
Tradeoff: an edit that only moves a value to a different line will not reappear as "new." Moved or duplicated values may still match by occurrence index. Commit a fresh baseline when you intentionally accept a new set of findings.
Output formats
Default output is a Rich console report (grouped by file → severity-colored table → totals).
piilint . --format json
piilint . --format sarif > piilint.sarif
piilint . --format json --baseline piilint-baseline.json --fail-on high
JSON includes a config_hash: SHA-256 of a canonical JSON snapshot of the effective
scan config fields that affect detection/policy (fail_on, min_confidence, exclude,
entity_enabled, severity_overrides, allowlists, phone_region). Paths and timestamps
are excluded so the hash is stable across identical policy runs.
--show-matches unmasks the console Sample column for local triage only. It is refused
when CI=true (exit 2) and does not apply to JSON/SARIF (those formats never emit raw PII).
Pre-commit hook
This repo ships a pre-commit hook definition in .pre-commit-hooks.yaml.
Add to your consuming project's .pre-commit-config.yaml:
repos:
- repo: https://github.com/thelonewander3r/PIIScanner
rev: v0.1.0 # pin to a release tag when available
hooks:
- id: piilint
# Default args from the hook repo: --staged --fail-on medium
# Override fail-on (or drop --staged) by replacing args:
# args: ["--staged", "--fail-on", "high"]
Notes:
- The hook runs
piilint --stagedand setspass_filenames: falseso pre-commit does not append paths (staged mode reads the git index). fail-ondefaults to medium in the hook; change viaargsas shown above.- Requires a git repository at hook time (same as CLI
--staged).
GitHub Action
Composite action at action.yml. Example workflow:
name: piilint
on:
pull_request:
push:
branches: [main]
jobs:
scan:
runs-on: ubuntu-latest
permissions:
contents: read
# needed only if you upload SARIF:
# security-events: write
steps:
- uses: actions/checkout@v4
- name: Run piilint
id: piilint
uses: thelonewander3r/PIIScanner@main # pin to a tag when available
with:
path: .
fail-on: high
format: sarif # console | json | sarif
# baseline: piilint-baseline.json
# staged: "false"
# version: "0.1.0" # install from PyPI; omit to pip-install action checkout
# extra-args: "--exclude 'vendor/**'"
# SARIF upload is the caller's job — the action only writes the file.
- name: Upload SARIF
if: always()
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: ${{ steps.piilint.outputs.sarif-path }}
Action inputs
| Input | Default | Description |
|---|---|---|
path |
. |
Path to scan |
fail-on |
(empty) | high / medium / low / never (empty → config/default) |
format |
console |
console / json / sarif |
baseline |
(empty) | Optional baseline JSON path |
staged |
false |
Scan only git-staged files |
extra-args |
(empty) | Extra CLI args (space-separated) |
version |
(empty) | PyPI version; empty installs from github.action_path |
sarif-file |
piilint.sarif |
Output path when format=sarif |
python-version |
3.12 |
Python for the composite runner |
Action outputs
| Output | Description |
|---|---|
sarif-path |
Path to written SARIF when format=sarif; empty otherwise |
CI & release
- CI:
.github/workflows/ci.yml—{ubuntu, windows, macos} × {3.10, 3.13}with ruff, mypy (files=src/piilint), pytest (incl. benchmark gate), andpiilint --version. - Release:
.github/workflows/release.yml— on tagv*/ GitHub Release published: build withuv build(hatchling), publish viapypa/gh-action-pypi-publishusing OIDC trusted publishing (no long-lived PyPI token).
PyPI trusted publisher checklist (maintainer)
Full tag-day steps: docs/RELEASE.md.
Hard stop: tag v0.1.0 only after Emanuel’s explicit go. No long-lived PyPI API tokens.
Emanuel-only (PyPI UI)
- Create project
piilintor register a pending trusted publisher (preferred; creates the project on first upload). - PyPI → Publishing → Trusted publishers → GitHub:
- Owner:
thelonewander3r - Repository:
PIIScanner - Workflow name:
release.yml(filename only) - Environment name:
pypi(must matchrelease.yml)
- Owner:
Emanuel-only (GitHub UI)
- Repo Settings → Environments → create
pypi(recommend required reviewers / wait timer).
Then (after Emanuel go)
- Do not store a PyPI API token in Actions secrets for this flow.
- From release commit on
main:git tag v0.1.0 && git push origin v0.1.0→ watch Actions Release → verifyuvx piilint --version/pipx install piilint.
Optional NER (names & addresses)
PERSON/ADDRESS detection is off by default and lives behind an optional extra so the base install stays lean and scan-time stays offline.
pip install "piilint[ner]" # or: uv sync --extra ner
piilint setup-ner # downloads en_core_web_sm (network; once)
piilint . --ner # enable PERSON + ADDRESS for this run
- Without
[ner]installed, normal scans are unchanged;piilint . --nerexits 2 with an install hint. - With
[ner]but no model,--nerexits 2 asking you to runsetup-ner. - Config toggles
entities.person/entities.addressdefault to false;--nerenables both for the run. - Enabling PERSON/ADDRESS via config alone still requires the
[ner]extra + model (same clear exit 2 guidance as--nerif they are missing). - Only English (
en_core_web_sm) is supported in this phase. No scan-time network — model download is setup-only.
Status
Phases 0–8 are complete, including Phase 7 optional NER (piilint[ner], setup-ner, --ner for PERSON/ADDRESS). Deterministic recognizers, text + tabular + notebook adapters, console / JSON / SARIF reporters, synthetic benchmark corpus + CI gate, config/policy/noise controls, baseline subtraction, --staged mode, CI/release workflows, pre-commit hook, GitHub Action, and launch docs. Sprint 7 prep targets first PyPI 0.1.0; package is not published until Emanuel go + v0.1.0 tag + OIDC trusted publisher (see docs/RELEASE.md).
Contributing & security
- Contributors: see
CONTRIBUTING.md - Vulnerability reports: see
SECURITY.md - Changelog:
CHANGELOG.md - Maintainer release runbook:
docs/RELEASE.md
License
Apache-2.0
Metadata
Release files for piilint 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| piilint-0.1.0.tar.gz | 70.7 kB | With provenance | Details for piilint-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| piilint-0.1.0-py3-none-any.whl | Python 3 | none | any | Details for piilint-0.1.0-py3-none-any.whl |
Total release size: 123.4 kB
Release files / piilint-0.1.0.tar.gz
| Download URL | piilint-0.1.0.tar.gz |
|---|---|
| Size | 70.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
d7a5ebf0da71013e5e70b2d1dc2379fca6e1e42de05b9c9e2841a987c608b2d4
|
|
BLAKE2b-256 checksum How to use checksums |
0a69326e511d8f8e704ae7e229a56f62a9a2288091e047ddc64da397b5624e76
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 12, 2026.
Transparency logRelease files / piilint-0.1.0-py3-none-any.whl
| Download URL | piilint-0.1.0-py3-none-any.whl |
|---|---|
| Size | 52.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
f410fc2d7e985b15d972d823347596010fde42e9f54ea24273532eb6ad2cec22
|
|
BLAKE2b-256 checksum How to use checksums |
a38e42bcaeec20169f7fd7be862de45ca26c3cfc2be6563c2bdb2803038298fa
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 12, 2026.
Transparency logSecurity for piilint 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for piilint-0.1.0.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 12, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.