---

📖 Table of Contents

• Overview
• Why SecretShield?
• Key Features
• How It Works
• Supported Detectors
• Installation
• Quick Start & Realistic Walkthrough
• CLI Reference & Commands
• Exit Codes
• Git Pre-Commit Hook
• Configuration (.secretshield.toml)
• False Positive Management
• Security Model
• Privacy Guarantee
• Limitations
• Development & Testing
• Roadmap
• License

---

🔍 Overview

SecretShield is a developer-first security CLI designed to scan files, staged changes, and Git commit history for exposed secrets and credentials before they are committed or pushed.

Operating 100% locally and offline, SecretShield ensures that your sensitive source code, API keys, and environment variables never leave your workstation.

                  ┌────────────────────────────────────────┐
                  │           Developer Changes            │
                  └───────────────────┬────────────────────┘
                                      │
                                      ▼
             ┌──────────────────────────────────────────────────┐
             │            SecretShield Scanner Engine           │
             │  • Pattern Matching (Regex)                      │
             │  • Shannon Entropy Calculation                   │
             │  • Contextual Keyword Analysis                   │
             │  • HMAC-SHA256 Fingerprinting                    │
             └────────────────────────┬─────────────────────────┘
                                      │
                   ┌──────────────────┴──────────────────┐
                   ▼                                     ▼
        [ ❌ Secrets Detected ]                [ ✅ Clean Repository ]
         Commit Blocked / Alert                 Safe to Commit & Push

---

💡 Why SecretShield?

Accidental credential exposure is one of the most common vectors for security breaches. Once a secret is pushed to a remote Git repository:

1. It is permanently in Git history, requiring complex history rewrites (git filter-repo) to purge.
2. Automated scrapers detect it in seconds, exploiting keys before alerts can be acknowledged.
3. Revocation is disruptive, causing service downtime and team churn.

SecretShield prevents leaks at the source—running as a local pre-commit hook or fast CI check to intercept credentials before they leave your computer.

---

✨ Key Features

• 🔒 100% Local & Offline: Zero network calls, zero tracking, zero external telemetry.
• ⚡ Blazing Fast: Optimized regex detectors with ReDoS protection and fast binary skipping.
• 🧠 Multi-Signal Scoring: Combines regex patterns, Shannon entropy analysis, and contextual code keywords to drastically minimize false positives.
• 🛡️ Zero Secret Exposure: Raw secrets are never saved, printed, or written to disk. All outputs are strictly redacted via a centralized masking engine.
• 🔑 Cryptographic Fingerprinting: Deterministic HMAC-SHA256 fingerprints allow you to safely ignore false positives without ever exposing or saving the underlying secret.
• 🪝 One-Click Pre-Commit Integration: Native Git hook installation (secretshield install) blocks accidental commits instantly.
• 📜 Full History & Staged Scanning: Scan unstaged working directories, staged git indexes, or complete commit histories.

---

🔎 How It Works

SecretShield evaluates suspicious tokens using a 4-pillar detection pipeline:

flowchart LR
    A[Input Content] --> B[Fast Regex Matcher]
    B --> C[Shannon Entropy]
    C --> D[Context Analyzer]
    D --> E[Scoring & Thresholding]
    E --> F{Confidence >= Threshold?}
    F -->|Yes| G[Redacted Masking & Finding Report]
    F -->|No| H[Ignore / Discard]

1. Regex Detectors: High-precision compiled regular expressions identify known token formats (e.g., GitHub PATs, AWS access keys, Discord webhooks).
2. Shannon Entropy: Measures randomness in candidate strings to catch high-entropy generic keys and passwords.
3. Context Analyzer: Inspects variable names, file names, comments, and assignment operators (e.g., boosting API_KEY = "..." while downgrading test_dummy = "...").
4. Masking & Fingerprinting: Redacts values to safe previews (https://discord.com/api/webhooks/123/********) and computes an HMAC fingerprint for allowlisting.

---

🎯 Supported Detectors

SecretShield includes out-of-the-box detectors for major cloud providers, SaaS platforms, and generic credentials:

Detector	Pattern Type	Default Severity	Description
Discord Webhook	https://discord.com/api/webhooks/...	HIGH	Discord incoming webhook URLs with active channel tokens
GitHub PAT (Classic & Fine-Grained)	ghp_..., github_pat_..., gho_...	CRITICAL	GitHub Personal Access Tokens and OAuth credentials
AWS Access Key & Secret	AKIA[0-9A-Z]{16}, aws_secret_access_key	CRITICAL	Amazon Web Services IAM access keys and secret keys
MongoDB Connection URI	mongodb(+srv)://user:pass@host	CRITICAL	Database connection strings with embedded passwords
JSON Web Tokens (JWT)	eyJ... . eyJ... . ...	MEDIUM	Base64-encoded signed authentication tokens
Private Keys	-----BEGIN RSA/EC/OPENSSH PRIVATE KEY-----	CRITICAL	SSH and TLS/SSL private key headers and cryptographic keys
Slack Tokens	xoxb-..., xoxp-..., xoxa-...	HIGH	Slack bot, user, and workspace tokens
Stripe API Keys	sk_live_..., rk_live_..., sk_test_...	CRITICAL	Stripe payment gateway secret and restricted keys
Google API Keys	AIza[0-9A-Za-z_-]{35}	HIGH	Google Cloud & Firebase public/restricted API keys
SendGrid API Keys	SG.[A-Za-z0-9_-]{22}.[A-Za-z0-9_-]{43}	HIGH	SendGrid transactional email service credentials
Twilio API Keys	SK[0-9a-fA-F]{32}	HIGH	Twilio communications API keys
Heroku API Keys	UUID format under heroku_api_key	HIGH	Heroku platform management API keys
Azure Connection Strings	AccountKey=..., SharedAccessKey=...	CRITICAL	Azure Storage & Service Bus shared access signatures
Generic API Keys & Tokens	High-entropy strings in key assignments	MEDIUM	api_key = "...", bearer_token = "..."
Passwords & Secrets	Assignment statements with credentials	MEDIUM	password = "...", client_secret = "..."

---

📦 Installation

Using pip

pip install secretshield

Using uv (Recommended for fast environments)

uv pip install secretshield

From Source

git clone https://github.com/secretshield/secretshield.git
cd secretshield
uv sync

---

🚀 Quick Start & Realistic Walkthrough

Let's see how SecretShield prevents accidental credential leaks.

1. The Scenario

Suppose a developer accidentally pastes a live Discord webhook into a configuration file src/config.py:

# src/config.py
DISCORD_ALERT_URL = "https://discord.com/api/webhooks/1234567890123456789/faketoken12345678901234567890123456789012345678901234567890abcdef"

2. Run a Scan

Run secretshield scan across your repository:

secretshield scan

3. Masked Findings Output

SecretShield immediately identifies the exposed credential, prints a masked preview, and exits with code 1:

============================== SecretShield Scan ==============================
Target: C:\Users\Yogeswar\development\Projects\secretshield
Scan Type: repository

[!] FINDINGS DETECTED (1 issue found):

  1. HIGH | Confidence: 95% (Very High)
     Type:        Discord Webhook
     Detector:    discord_webhook
     Location:    src/config.py:2:22
     Masked:      https://discord.com/api/webhooks/1234567890123456789/********
     Fingerprint: c8f31b0a9e2d4174

--------------------------------------------------------------------------------
Summary: 1 finding(s) in 14 file(s) scanned.
Critical: 0 | High: 1 | Medium: 0 | Low: 0
================================================================================

[!IMPORTANT] Notice how the sensitive authentication token faketoken123... is automatically redacted to ********. SecretShield guarantees that raw credentials are never printed to terminal screens or saved to log files.

---

💻 CLI Reference & Commands

secretshield scan [PATH]

Scans the current workspace or specified directory.

# Scan current repository
secretshield scan

# Scan a specific directory
secretshield scan ./src

# Scan only staged Git files (fast pre-commit check)
secretshield scan --staged

# Scan entire Git commit history for past leaks
secretshield scan --history

# Output findings in JSON format for automated tooling
secretshield scan --json

# CI Mode: minimal quiet output with strict exit codes
secretshield scan --ci

secretshield install

Installs SecretShield as a native Git pre-commit hook in .git/hooks/pre-commit.

secretshield install
# Successfully installed SecretShield pre-commit hook.

secretshield uninstall

Removes the SecretShield hook from .git/hooks/pre-commit.

secretshield uninstall
# Successfully uninstalled SecretShield pre-commit hook.

secretshield ignore <FINGERPRINT>

Adds a finding's HMAC fingerprint to .secretshield.toml allowlist to suppress future alerts on verified false positives.

secretshield ignore c8f31b0a9e2d4174
# Fingerprint 'c8f31b0a9e2d4174' added to allowlist in .secretshield.toml

secretshield version

Displays current version and detector environment information.

secretshield version
# SecretShield v0.1.0

---

⚙️ CLI Options & Flags

Flag	Commands	Description
--staged	scan	Scan only Git staged files (modified in index)
--history	scan	Scan past Git commits for historically leaked secrets
--json	scan	Emit structured JSON output to stdout
--ci	scan	CI mode: non-interactive, streamlined logging
--severity <LEVEL>	scan	Minimum severity filter (LOW, MEDIUM, HIGH, CRITICAL)
--confidence <0-100>	scan	Minimum confidence score threshold (default: 0)

---

🚦 Exit Codes

SecretShield uses standardized exit codes suitable for CI/CD pipelines and Git hook runners:

Exit Code	Meaning	Description
0	Clean	No secret findings detected. All files passed checks.
1	Findings	One or more active secret findings were detected.
2	Error	Runtime error, missing Git repository, or invalid configuration.

---

🪝 Git Pre-Commit Hook

Install the automated pre-commit hook with one command:

secretshield install

When you run git commit, the hook automatically runs secretshield scan --staged:

• If no secrets are staged, the commit proceeds smoothly.
• If a secret is staged, the commit is blocked, displaying the file, line, and masked credential so you can remove it before sharing code.

To temporarily bypass the hook in emergencies:

git commit --no-verify -m "Emergency hotfix"

---

🛠️ Configuration (.secretshield.toml)

SecretShield can be configured per-repository via a .secretshield.toml file in the root directory:

# SecretShield Configuration
# Place this file in the root of your repository.

[secretshield]
# Minimum severity to alert on: "LOW", "MEDIUM", "HIGH", "CRITICAL"
severity_threshold = "LOW"

# Minimum confidence score (0-100) to report
confidence_threshold = 0

# Maximum file size to scan in bytes (default: 10 MB)
max_file_size = 10485760

# Whether to attempt scanning binary files (default: false)
scan_binary = false

# Whether pre-commit hook should abort commits when findings are detected (default: true)
block_on_findings = true

[secretshield.ignored_paths]
# Globs and relative directory paths to exclude from scanning
paths = [
    "tests/fixtures/**",
    "docs/**",
    "vendor/**",
]

[secretshield.allowlist]
# One-way HMAC fingerprints of known/reviewed false positives
fingerprints = [
    "c8f31b0a9e2d4174",
    "9a12b4e87c53d102",
]

---

🛡️ False Positive Management

SecretShield provides a safe mechanism for handling false positives without storing or committing raw secrets.

How Fingerprints Work

When SecretShield detects a finding, it generates a 16-character HMAC-SHA256 fingerprint: $$\text{Fingerprint} = \text{HMAC-SHA256}(\text{Key}=\text{"secretshield-fingerprint-v1"}, \text{Message}=\text{Detector} \parallel \text{Path} \parallel \text{Line} \parallel \text{Secret})[0..16]$$

• Deterministic: Running scans on the same file and line produces the exact same fingerprint.
• One-Way / Non-Reversible: You cannot reverse the fingerprint to reconstruct the secret.
• Safe to Commit: The .secretshield.toml allowlist can be checked into version control without leaking the secret.

To ignore a finding:

secretshield ignore <fingerprint>

---

🔒 Security Model

SecretShield is built under a Zero-Trust Local Execution architecture:

1. No Network Activity: SecretShield makes 0 HTTP/network requests. It works in air-gapped environments.
2. Masking at the Core: Raw secret strings never escape the detector boundary. Only masked strings and one-way hashes are passed to CLI formatters and JSON builders.
3. No Unsafe Code Execution: Repository content is inspected as passive text data. No eval(), exec(), or dynamic imports are performed.
4. Command Injection Immunized: Git interactions use parameterized array arguments with shell=False.
5. Path Traversal Guard: All file resolutions verify boundary confinement to the repository root.

For full details, review our SECURITY.md.

---

🛡️ Privacy Guarantee

• Your code stays on your machine: SecretShield will never transmit file contents, metadata, or telemetry to external servers.
• No Analytics / No Telemetry: We collect zero usage statistics, crash beacons, or IP addresses.
• Auditable: All scanning and masking logic is open source and inspectable under the MIT License.

---

⚠️ Limitations

• Obfuscated & Encrypted Secrets: SecretShield detects plaintext and standard base64/hex representations. It cannot detect secrets obfuscated via multi-stage encryption or custom runtime decoders.
• Hook Bypasses: Pre-commit hooks can be bypassed using git commit --no-verify. We recommend running secretshield scan --ci in your CI/CD pipelines (e.g., GitHub Actions) as a secondary defense layer.
• Multi-Line Delimited Secrets: Extremely fragmented tokens broken across many non-standard lines may receive lower confidence scores.

---

🛠️ Development & Testing

SecretShield is built with Python 3.12+ and managed with uv.

Development Setup

# Clone the repository
git clone https://github.com/secretshield/secretshield.git
cd secretshield

# Install dependencies and sync virtual environment
uv sync

# Run code linters and security checks
uv run ruff check
uv run ruff format --check

Running Tests

# Run test suite with verbose output
uv run pytest -v

# Run tests with code coverage report
uv run pytest --cov=secretshield --cov-report=term-missing

---

🗺️ Roadmap

• Web Dashboard: Optional local web UI (secretshield ui) for visual findings review.
• IDE Extensions: Real-time secret flagging extension for VS Code and JetBrains IDEs.
• Custom Rule Plugins: Support for user-defined TOML regex patterns and custom detectors.
• GitHub Action: Official native GitHub Action with PR annotations and SARIF export.
• Historical Git Bisection: Automated bisecting to identify the exact commit author and date a secret was introduced.

---

📄 License

SecretShield is released under the GNU General Public License v3.0.
Copyright (c) 2024 SecretShield Contributors.