dlp-patterns 0.1.0

Fast, zero-dependency DLP pattern scanner — detects PII, secrets, and sensitive data in text

dlp-patterns

Fast, zero-dependency DLP pattern scanner for Python.

Detects PII, secrets, and sensitive data in any text — documents, logs, source code, emails. Built from the scanning engine that powers Spidercob, an enterprise DLP platform.

import dlp_patterns

result = dlp_patterns.scan("My SSN is 432-78-9012 and CC 4111 1111 1111 1111")
print(result.highest_severity)   # CRITICAL
print(result.critical[0].type)   # credit_card

clean = dlp_patterns.redact("Send to [e-mail] with CC 4111 1111 1111 1111")
# "Send to [REDACTED: Email Address] with [REDACTED: Credit Card Number]"

Install

pip install dlp-patterns

No external dependencies. Python 3.9+.

What it detects

Category	Patterns
Financial	Credit cards (Luhn + BIN), SSN, IBAN, bank account, routing number
PII	Email, US phone, passport, driver's license, date of birth
Healthcare	Medical record numbers, ICD-10 codes, NPI, DEA numbers, NDC codes
Secrets	AWS keys, GitHub PATs, Slack tokens, Google API keys, Bearer tokens, JWTs
Cloud / SaaS	Stripe, SendGrid, Mailgun, Twilio, HuggingFace, NPM, Cloudflare, Azure
Infrastructure	DB connection strings, hardcoded passwords, Docker registry auth
Crypto	RSA/EC/SSH/PGP private keys, X.509 certs
Webhooks	Slack webhooks, Discord webhooks, Telegram bot tokens
Cryptocurrency	Bitcoin addresses, Ethereum addresses

50+ pattern categories total.

Features

• Validators — Luhn check for credit cards, FICA rules for SSNs, JSON decode for JWTs. Reduces false positives before they reach you.
• Entropy gating — Shannon entropy + sliding-window analysis rejects low-entropy matches (e.g. aaaaaaa...) from generic secret patterns.
• Context scoring — Each finding gets a context_score (0–1) based on surrounding words. Proximity to production, secret, deploy boosts the score; proximity to example, placeholder, test lowers it.
• Required context keywords — Patterns like ICD-10 codes and Telegram tokens only fire when relevant keywords appear nearby.
• secrets_only mode — Scan just for API keys and credentials, skipping PII. Faster for CI/CD secret scanning.
• redact() — Replace all findings with [REDACTED: <type>].
• fuzz() — Replace findings with realistic fake values (requires faker). Useful for building safe test datasets from production data.
• CLI — dlp-scan command for shell pipelines and CI.

Usage

Python API

import dlp_patterns

# Scan
result = dlp_patterns.scan(text)

result.has_findings          # bool
result.highest_severity      # "CRITICAL" | "HIGH" | "MEDIUM" | "LOW" | None
result.critical              # list[Finding]
result.all                   # all findings across severities
result.elapsed_ms            # scan time in milliseconds

# Each Finding:
f = result.critical[0]
f.type                       # "credit_card"
f.description                # "Credit Card Number"
f.value                      # masked: "4111...1111"
f.severity                   # "CRITICAL"
f.position                   # "char 10-29"
f.context                    # surrounding text (±100 chars)
f.context_score              # float 0.0–1.0
f.context_keywords_found     # ["payment", "billing"]

# Secrets only (faster for source code scanning)
result = dlp_patterns.scan(code, secrets_only=True)

# Redact
clean = dlp_patterns.redact(text)

# Fuzz (pip install dlp-patterns[fuzz])
safe = dlp_patterns.fuzz(text)

# JSON output
result.to_dict()

CLI

# Scan a string
dlp-scan "My SSN is 432-78-9012"

# Scan a file
dlp-scan path/to/document.txt

# Pipe from stdin
cat logfile.txt | dlp-scan

# JSON output
dlp-scan --json document.txt

# Redact in place
dlp-scan --redact document.txt > clean.txt

# Secrets only (for source code)
dlp-scan --secrets-only src/config.py

# Exit code: 1 if CRITICAL findings, 0 otherwise — useful in CI
dlp-scan --secrets-only . && echo "clean"

Use in CI (GitHub Actions)

- name: DLP secret scan
  run: |
    pip install dlp-patterns
    dlp-scan --secrets-only --json src/ | tee dlp-report.json

Advanced — use Scanner directly

from dlp_patterns import Scanner

scanner = Scanner()

# Reuse the same instance (compiled patterns cached)
for text in documents:
    result = scanner.scan(text)
    if result.has_findings:
        print(result.to_dict())

Enterprise

Need a full DLP platform with dashboards, audit logs, ICAP proxy integration, Gmail/Slack scanning, compliance reports, and AI-powered analysis?

→ Spidercob — the enterprise DLP platform this library is extracted from.

License

Apache 2.0 — free for commercial use.