entro-scan 1.2.3

Entropy-based secret scanner for source code — detects API keys, tokens, passwords, and other sensitive data leaks

entro-scan

Entropy-based secret scanner for source code — detects API keys, tokens, passwords, and other sensitive data leaks before they reach production.

---

Demo

Features

• AI-triaged scanning — local qwen2.5 LLM classifies each finding as REAL or NOISE; generates pattern-specific remediation advice (rotate, restrict, move to env var, etc.)
• Shannon entropy analysis — finds high-entropy strings that look like secrets
• 60+ provider patterns — AWS (incl. session tokens), GitHub (classic, fine-grained, OAuth, app install), GitLab, Slack (bot + webhook), Discord (bot + webhook), Telegram bot, Stripe (live, test, publishable, webhook), GCP service-account JSON, Azure Storage / SAS / client secret, OpenAI (sk-/sk-proj-/sk-svcacct-), Anthropic (sk-ant-), Supabase service role + JWT, Vercel, Clerk (publishable + secret), Linear, Notion, Figma, HuggingFace, PyPI, Shopify, PlanetScale, Netlify, Asana, Atlassian, Twilio, SendGrid, Mailgun, Mailchimp, Dropbox, Square, Heroku, database connection URLs with embedded credentials, private keys, and more
• Confidence + risk scoring — each finding has severity, confidence (0-100), and a composite risk_score; gate CI on --min-confidence
• Finding fingerprints — stable 16-char IDs for dedup, baselining, and SARIF partialFingerprints
• Git history / staged / diff scanning — pre-commit ready, with --diff-filter=ACMR
• Reporters — terminal, JSON, CSV, SARIF 2.1.0 (with security-severity), Markdown (PR-ready), and GitHub Actions workflow-command annotations
• GitHub-native integration — --pr-comment (create-or-update with marker), --github-step-summary, --github-annotations, auto-detected inside Actions
• Triage workflows — explain findings by fingerprint, install pre-commit hooks, generate .env.example placeholders, group monorepo results, and run safe provider verification
• Dashboard/editor outputs — single-file HTML dashboard and VS Code problem-matcher friendly output
• Low false-positive rate — filters for placeholders, UUIDs, hex blobs, regex source, template/format strings, URLs without credentials, and inline # entro-scan: ignore directives (ignore, ignore-line, ignore-next-line)
• Smart file walker — .gitignore honored, binary sniffer, configurable max_file_size_mb, generated reports (scan.json, scan.sarif, baselines) auto-skipped
• Redacted baselines — by default baselines store sha256 + fingerprint only; opt-in raw storage with --no-redact-baseline
• Configurable — TOML config (.entro-scan.toml or [tool.entro-scan] in pyproject.toml)
• Zero dependencies — pure Python 3.11+ standard library, including the GitHub API client
• Pre-commit hook ready — --git-staged --fail-on-severity critical
• CI/CD friendly — --fail-on-severity, exit codes 0/1/2

---

Installation

Recommended: pipx

Install entro-scan as an isolated global CLI tool.

pipx install entro-scan
entro-scan --help

Python / pip

pip install entro-scan

npm / npx

npx entro-scan

The npm package currently acts as a lightweight wrapper/helper around the Python CLI version.

Install from source

git clone https://github.com/vyofgod/entro-scan.git
cd entro-scan
pip install -e .

---

Usage

# Scan current directory
entro-scan

# Scan a specific path
entro-scan /path/to/project

# Custom entropy threshold (lower = more findings)
entro-scan /path --threshold 4.0

# Output in JSON format
entro-scan /path --format json

# Output to file
entro-scan /path --format json -o results.json

# Scan git history (last 100 commits)
entro-scan /path --git

# Scan git history with custom depth
entro-scan /path --git --max-commits 500

# Scan only staged git files (--staged alias)
entro-scan /path --staged

# Scan only modified git files (--diff alias)
entro-scan /path --diff

# Parallel scan with 8 workers
entro-scan /path --workers 8

# Quiet mode (only findings, no banner)
entro-scan /path --quiet

# Mask secrets in output (default: true, --no-mask-secrets to disable)
entro-scan /path --mask-secrets

# Show unmasked secrets
entro-scan /path --no-mask-secrets

# Fail CI if critical findings are found (--fail-on-findings alias)
entro-scan /path --fail-on-severity critical

# Use a baseline file to ignore known findings
entro-scan /path --baseline .entro-scan.baseline.json

# Save current findings as a new baseline (--update-baseline alias)
entro-scan /path --save-baseline .entro-scan.baseline.json

# Generate a default config file
entro-scan --init

# Markdown PR-style report
entro-scan . --format markdown -o entro-scan-report.md

# Single-file HTML dashboard report
entro-scan . --format html -o entro-scan-report.html

# VS Code / problem matcher friendly output
entro-scan . --format vscode

# GitHub Actions: annotations + PR comment + step summary (all auto-on in Actions)
entro-scan . --github-annotations --github-step-summary --pr-comment

# Filter low-confidence noise
entro-scan . --min-confidence 70

# Disable .gitignore honoring
entro-scan . --no-use-gitignore

# Skip files larger than 1 MB
entro-scan . --max-file-size 1

# Apply scan profiles
entro-scan . --profile ci
entro-scan . --profile paranoid
entro-scan . --profile frontend
entro-scan . --profile backend

# Only report findings not present in a baseline
entro-scan . --only-new --baseline .entro-scan.baseline.json

# Attach git blame / first-seen metadata
entro-scan . --blame --format json

# Group monorepo findings
entro-scan . --group-by package

# Verify supported provider tokens (GitHub, OpenAI, Slack bot tokens)
entro-scan verify .

AI-Triaged Scanning

New in v1.2.0: Use the local AI model to automatically classify findings and get pattern-specific recommendations.

# Setup: check system requirements and download qwen2.5:0.5b (398 MB)
entro-scan ai

# Scan with AI triage (REAL vs NOISE) + pattern-specific advice
entro-scan ai /path/to/project

# Force model re-download or setup
entro-scan ai --setup

The AI triage:

• Detects common false positives (CSS values, test files, minified JS, placeholders)
• Classifies findings as REAL (rotate) or NOISE (ignore)
• Generates actionable recommendations per pattern (e.g., "Rotate Google API key in Cloud Console, restrict by referrer")
• Analyzes repo context to infer project type and credential risk level

Requires Ollama + ~1 GB free RAM (install automatically on first run).

Inline ignore directives

Add a comment on the same line, or the line above, to suppress a finding:

TOKEN = "ghp_..." # entro-scan: ignore
# entro-scan: ignore-next-line
TOKEN = "ghp_..."

Supported comment leaders: #, //, /* */, --.

Workflow Commands

# Explain why a finding was reported
entro-scan explain abc123def4567890 .

# Include first-seen git metadata and safe provider verification in the explanation
entro-scan explain abc123def4567890 . --blame --verify

# Review findings with remediation and fingerprints
entro-scan fix .

# Accept current findings into a redacted baseline
entro-scan fix . --action baseline --baseline .entro-scan.baseline.json

# Add placeholder keys to .env.example for detected providers
entro-scan fix . --action env-example

# Install a pre-commit hook that scans staged files
entro-scan install-hook

---

Exit Codes

Code	Meaning
0	Success, no findings or findings don't meet severity threshold
1	Error (invalid config, file not found, etc.)
2	Blocking findings detected

---

Output Formats

Terminal (default)

Color-coded output with severity levels:

• Red (score > 4.5): Critical — likely a secret
• Yellow (score > 3.9): High — suspicious
• Green (score <= 3.9): Medium — low-confidence finding

JSON

Machine-readable output for CI/CD pipelines.

CSV

Spreadsheet-friendly output for reporting.

SARIF

Static Analysis Results Interchange Format 2.1.0 — compatible with GitHub code scanning. Each rule includes a security-severity property and each result carries a stable partialFingerprints entry so GitHub deduplicates findings across scans.

Markdown

Pull-request friendly summary with severity badges, a table, and a remediation appendix. Used by --pr-comment and --github-step-summary.

HTML

Single-file dashboard with severity cards, filter buttons, remediation text, and optional verification / first-seen metadata.

VS Code

Problem-matcher friendly file:line:column: severity: message output for editor and CI log integrations.

GitHub Annotations

Emits ::error file=...:: workflow commands so GitHub renders inline annotations on the PR diff. Auto-enabled when running inside Actions, or via --github-annotations.

---

Configuration

Create .entro-scan.toml in your project root:

threshold = 3.5
workers = 2
quiet = false
verbose = false
output_format = "terminal"
git_enabled = false
git_staged = false
git_diff = false
max_commits = 100
mask_secrets = true
# fail_on_severity = "critical"  # Options: critical, high, medium, any
# baseline_path = ".entro-scan.baseline.json"

exclude_dirs = [
    ".git", "node_modules", "venv", "__pycache__",
    ".idea", ".vscode", "build", "dist", "target",
]

exclude_files = [
    "package-lock.json", "yarn.lock", "pnpm-lock.yaml",
    "cargo.lock", "go.sum", "poetry.lock",
]

include_extensions = [
    ".py", ".rs", ".js", ".ts", ".go", ".java", ".kt", ".swift",
    ".rb", ".php", ".sh", ".json", ".yaml", ".yml", ".toml", ".env",
]

# Allowlist patterns (text containing these will be ignored)
# allowlist_patterns = [
#     "test-secret",
#     "example-key",
# ]

# Allowlist hashes (sha256 of the secret text)
# allowlist_hashes = [
#     "abc123...",
# ]

Alternatively, config can live under [tool.entro-scan] in your pyproject.toml.

---

Supported Patterns

Pattern	Severity
JWT (JSON Web Tokens)	Critical
AWS Access Key ID	Critical
AWS Secret Key	Critical
GitHub Token	Critical
Slack Token	Critical
Private Keys (RSA/DSA/EC/OpenSSH)	Critical
Stripe API Key	Critical
Mailchimp API Key	Critical
SendGrid API Key	Critical
Dropbox API Key	Critical
PayPal Braintree Access Token	Critical
NPM Token	Critical
Docker Hub Token	Critical
OpenAI API Key	Critical
Anthropic API Key	Critical
Supabase Key	Critical
Vercel Token	Critical
Linear API Key	Critical
Facebook Access Token	Critical
GitLab Token	High
Heroku API Key	High
Database URLs (Postgres, MySQL, MongoDB, Redis, SQLite, MariaDB, Oracle)	High
Square Access Token	High
Square OAuth Secret	High
Twitter API Key	High
Twitter Access Token	High
Google API Key	High
Twilio API Key	High
Twilio Account SID	High
Basic Auth	High
Clerk Key	High
API Key in URL	Medium
Generic API Keys / Secrets	Medium

---

Pre-commit Hook

Add to your .pre-commit-config.yaml:

repos:
  - repo: https://github.com/vyofgod/entro-scan
    rev: v1.0.0
    hooks:
      - id: entro-scan
        args: ["--staged", "--fail-on-severity", "critical"]

---

CI/CD Integration

GitHub Actions

Create .github/workflows/entro-scan.yml:

name: Secret Scan

on:
  push:
    branches: [main, master]
  pull_request:
    branches: [main, master]

jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.11"
          
      - name: Install entro-scan
        run: pip install entro-scan
        
      - name: Run entro-scan
        run: |
          entro-scan . \
            --format sarif \
            -o entro-scan-results.sarif \
            --fail-on-severity critical
            
      - name: Upload SARIF to GitHub
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: entro-scan-results.sarif

Or use the official GitHub Action — it wires PR comments, annotations, the step summary, and SARIF upload in one step:

name: Secret Scan
on:
  pull_request:
  push:
    branches: [main]

permissions:
  contents: read
  pull-requests: write   # for --pr-comment
  security-events: write # for SARIF upload to code scanning

jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0   # full history for --git
      - name: Run entro-scan
        uses: vyofgod/entro-scan@v1
        with:
          fail-on-severity: high
          format: sarif
          output: entro-scan.sarif
          pr-comment: "true"
          github-annotations: "true"
          step-summary: "true"
          upload-sarif: "true"

The action exports findings_count and critical_count outputs you can branch on in downstream steps.

---

Development

# Install dev dependencies
pip install pytest ruff

# Run tests
pytest tests/ -v

# Lint
ruff check .

# Type check
mypy entro_scan/

---

Why entropy?

Secrets like API keys, tokens, and passwords are typically random strings with high entropy (information density). Natural language text and code identifiers have much lower entropy.

By measuring the Shannon entropy of strings in your codebase, entro-scan can flag potential secrets with high accuracy.

---

License

MIT — see LICENSE for details.