Skip to main content

piilint

CI

Local-first PII scanner for the files developers actually commit and send — notebooks, CSV, JSON, Parquet, and source code. Nothing leaves your machine.

Disclaimer: piilint helps you find sensitive data before it leaks. It is a detection aid, not a compliance certification, and cannot guarantee that all sensitive data is found. It does not make anyone GDPR/HIPAA/PCI compliant.

Package: piilint (PyPI name piiscan was already taken) · License: Apache-2.0 · Repo: thelonewander3r/PIIScanner

Not a secrets scanner. Pair with gitleaks or trufflehog for API keys and tokens.


Five-minute path

1. Install

Once published to PyPI (after v0.1.0 — not yet released):

# recommended for CLI use
pipx install piilint
# or one-off
uvx piilint --version
# or classic
pip install piilint

Until then, install from git or a local checkout (package is not on PyPI yet):

pipx install git+https://github.com/thelonewander3r/PIIScanner.git
# or
uv sync --extra dev
uv run piilint --version

2. Scan

piilint .                     # scan the current directory
piilint . --fail-on high      # fail CI/pre-commit on high-severity findings

Exit codes: 0 clean / nothing staged · 1 findings at or above --fail-on · 2 usage/config/git error.

3. Wire into git / CI (optional)

Pre-commit — add to .pre-commit-config.yaml:

repos:
  - repo: https://github.com/thelonewander3r/PIIScanner
    rev: v0.1.0   # pin to a release tag when available
    hooks:
      - id: piilint
        # Default: --staged --fail-on medium

GitHub Action + SARIF — drop into a workflow:

- uses: actions/checkout@v4
- name: Run piilint
  id: piilint
  uses: thelonewander3r/PIIScanner@main   # pin to a tag when available
  with:
    path: .
    fail-on: high
    format: sarif
- name: Upload SARIF
  if: always()
  uses: github/codeql-action/upload-sarif@v3
  with:
    sarif_file: ${{ steps.piilint.outputs.sarif-path }}

4. Adopt without boiling the ocean

piilint baseline . -o piilint-baseline.json   # fingerprints only — never raw PII
piilint . --baseline piilint-baseline.json    # report NEW findings only
piilint . --staged                            # scan only git-staged files

Demo

The classic leak: a notebook runs df.head() and the output cell still holds customer rows when you commit the .ipynb.

Synthetic demo (no real PII): tests/corpus/notebook/leak_demo.ipynb

piilint tests/corpus/notebook

See also examples/README.md for a short pointer and expected story.

All of tests/corpus/ is 100% synthetic labeled data generated for tests.


Configuration

Precedence (highest wins): CLI flags → piilint.toml at the scan root → [tool.piilint] in pyproject.toml → built-in defaults.

# piilint.toml
[scan]
fail_on = "high"
min_confidence = 0.6
exclude = ["tests/fixtures/**"]

[entities]
ip_address = false
[entities.email]
severity = "medium"

[allowlist]
values  = ["support@mycompany.com"]
domains = ["example.com", "mycompany.dev"]
  • .piiignore — gitignore-syntax path excludes (combined with .gitignore).
  • Inline suppressions (text/code lines): # piilint: ignore or # piilint: ignore[EMAIL] (comma-list). Not applied to tabular/column-aggregated findings in v0.
  • Allowlists — exact normalized values and email domains drop matching findings.
  • Test-data downweight — obvious fixtures (example.com, 555-01xx, 4111…, RFC5737 IPs) get −0.4 confidence and severity capped at low, then min_confidence is re-applied.

Baseline + staged

Adopt without fixing history first, and scan only what is about to land in git.

piilint baseline . -o piilint-baseline.json
piilint . --baseline piilint-baseline.json
piilint . --staged

Fingerprint design: SHA-256(relative path, entity, normalized-value hash, occurrence index). Line numbers are excluded so ordinary edits do not resurrect old findings.

Tradeoff: an edit that only moves a value to a different line will not reappear as "new." Moved or duplicated values may still match by occurrence index. Commit a fresh baseline when you intentionally accept a new set of findings.


Output formats

Default output is a Rich console report (grouped by file → severity-colored table → totals).

piilint . --format json
piilint . --format sarif > piilint.sarif
piilint . --format json --baseline piilint-baseline.json --fail-on high

JSON includes a config_hash: SHA-256 of a canonical JSON snapshot of the effective scan config fields that affect detection/policy (fail_on, min_confidence, exclude, entity_enabled, severity_overrides, allowlists, phone_region). Paths and timestamps are excluded so the hash is stable across identical policy runs.

--show-matches unmasks the console Sample column for local triage only. It is refused when CI=true (exit 2) and does not apply to JSON/SARIF (those formats never emit raw PII).


Pre-commit hook

This repo ships a pre-commit hook definition in .pre-commit-hooks.yaml.

Add to your consuming project's .pre-commit-config.yaml:

repos:
  - repo: https://github.com/thelonewander3r/PIIScanner
    rev: v0.1.0   # pin to a release tag when available
    hooks:
      - id: piilint
        # Default args from the hook repo: --staged --fail-on medium
        # Override fail-on (or drop --staged) by replacing args:
        # args: ["--staged", "--fail-on", "high"]

Notes:

  • The hook runs piilint --staged and sets pass_filenames: false so pre-commit does not append paths (staged mode reads the git index).
  • fail-on defaults to medium in the hook; change via args as shown above.
  • Requires a git repository at hook time (same as CLI --staged).

GitHub Action

Composite action at action.yml. Example workflow:

name: piilint
on:
  pull_request:
  push:
    branches: [main]

jobs:
  scan:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      # needed only if you upload SARIF:
      # security-events: write
    steps:
      - uses: actions/checkout@v4

      - name: Run piilint
        id: piilint
        uses: thelonewander3r/PIIScanner@main   # pin to a tag when available
        with:
          path: .
          fail-on: high
          format: sarif          # console | json | sarif
          # baseline: piilint-baseline.json
          # staged: "false"
          # version: "0.1.0"     # install from PyPI; omit to pip-install action checkout
          # extra-args: "--exclude 'vendor/**'"

      # SARIF upload is the caller's job — the action only writes the file.
      - name: Upload SARIF
        if: always()
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: ${{ steps.piilint.outputs.sarif-path }}

Action inputs

Input Default Description
path . Path to scan
fail-on (empty) high / medium / low / never (empty → config/default)
format console console / json / sarif
baseline (empty) Optional baseline JSON path
staged false Scan only git-staged files
extra-args (empty) Extra CLI args (space-separated)
version (empty) PyPI version; empty installs from github.action_path
sarif-file piilint.sarif Output path when format=sarif
python-version 3.12 Python for the composite runner

Action outputs

Output Description
sarif-path Path to written SARIF when format=sarif; empty otherwise

CI & release

PyPI trusted publisher checklist (maintainer)

Full tag-day steps: docs/RELEASE.md.
Hard stop: tag v0.1.0 only after Emanuel’s explicit go. No long-lived PyPI API tokens.

Emanuel-only (PyPI UI)

  1. Create project piilint or register a pending trusted publisher (preferred; creates the project on first upload).
  2. PyPI → Publishing → Trusted publishers → GitHub:
    • Owner: thelonewander3r
    • Repository: PIIScanner
    • Workflow name: release.yml (filename only)
    • Environment name: pypi (must match release.yml)

Emanuel-only (GitHub UI)

  1. Repo Settings → Environments → create pypi (recommend required reviewers / wait timer).

Then (after Emanuel go)

  1. Do not store a PyPI API token in Actions secrets for this flow.
  2. From release commit on main: git tag v0.1.0 && git push origin v0.1.0 → watch Actions Release → verify uvx piilint --version / pipx install piilint.

Optional NER (names & addresses)

PERSON/ADDRESS detection is off by default and lives behind an optional extra so the base install stays lean and scan-time stays offline.

pip install "piilint[ner]"    # or: uv sync --extra ner
piilint setup-ner             # downloads en_core_web_sm (network; once)
piilint . --ner               # enable PERSON + ADDRESS for this run
  • Without [ner] installed, normal scans are unchanged; piilint . --ner exits 2 with an install hint.
  • With [ner] but no model, --ner exits 2 asking you to run setup-ner.
  • Config toggles entities.person / entities.address default to false; --ner enables both for the run.
  • Enabling PERSON/ADDRESS via config alone still requires the [ner] extra + model (same clear exit 2 guidance as --ner if they are missing).
  • Only English (en_core_web_sm) is supported in this phase. No scan-time network — model download is setup-only.

Status

Phases 0–8 are complete, including Phase 7 optional NER (piilint[ner], setup-ner, --ner for PERSON/ADDRESS). Deterministic recognizers, text + tabular + notebook adapters, console / JSON / SARIF reporters, synthetic benchmark corpus + CI gate, config/policy/noise controls, baseline subtraction, --staged mode, CI/release workflows, pre-commit hook, GitHub Action, and launch docs. Sprint 7 prep targets first PyPI 0.1.0; package is not published until Emanuel go + v0.1.0 tag + OIDC trusted publisher (see docs/RELEASE.md).

Contributing & security

License

Apache-2.0

Metadata

Release files for piilint 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for piilint 0.1.0
File Size Uploaded Provenance Actions
piilint-0.1.0.tar.gz 70.7 kB With provenance Details for piilint-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for piilint 0.1.0
File Interpreter ABI Platform Actions
piilint-0.1.0-py3-none-any.whl Python 3 none any Details for piilint-0.1.0-py3-none-any.whl

Total release size: 123.4 kB

Release files / piilint-0.1.0.tar.gz

Download URL piilint-0.1.0.tar.gz
Size 70.7 kB
Tags Source
SHA-256 checksum
How to use checksums
d7a5ebf0da71013e5e70b2d1dc2379fca6e1e42de05b9c9e2841a987c608b2d4
BLAKE2b-256 checksum
How to use checksums
0a69326e511d8f8e704ae7e229a56f62a9a2288091e047ddc64da397b5624e76
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Aug 12, 2026.

Transparency log

Release files / piilint-0.1.0-py3-none-any.whl

Download URL piilint-0.1.0-py3-none-any.whl
Size 52.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
f410fc2d7e985b15d972d823347596010fde42e9f54ea24273532eb6ad2cec22
BLAKE2b-256 checksum
How to use checksums
a38e42bcaeec20169f7fd7be862de45ca26c3cfc2be6563c2bdb2803038298fa
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Aug 12, 2026.

Transparency log

Security for piilint 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for piilint-0.1.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Aug 12, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page