harpocrates 0.3.1

ML-powered secrets detection tool

Harpocrates

ML-powered secrets detection that catches what regex can't see

---

AI coding tools leak secrets at 2× the rate of human-written code — 3.2% vs 1.6% of commits. 29 million secrets were exposed in 2025, up 34% year over year. One misconfigured environment variable cost a team $87k in a single night.

Regex scanners look for AWS_ACCESS_KEY_ID and GITHUB_TOKEN. They miss client_secret, ENCRYPTION_KEY, and API_SECRET — the names developers actually use. They miss secrets buried in comments. They miss env-var fallbacks. They miss the code your AI coding assistant just generated.

Harpocrates catches what slips through.

---

Harpocrates vs TruffleHog

Benchmarked on a 5,000-line held-out evaluation set, cross-referenced against TruffleHog's output using 1,430 TruffleHog findings as ground truth.

	TruffleHog	Harpocrates
Unique lines flagged	606	1,300
Lines only this tool caught	449	1,143
Lines both tools caught	157	157
Recall (held-out set)	—	97.3%
Precision (held-out set)	—	87.1%
Detection approach	Regex + live API verification	Regex + entropy + ML context
Scans comments	❌	✅
Catches ambiguous variable names	❌	✅
Catches env-var fallback leaks	❌	✅
Catches AI-scaffolded credential patterns	❌	✅
Classifies secret type (password, api_token, jwt…)	❌	✅
Verifies credentials are live via API calls	✅	❌

Harpocrates caught 1,143 lines TruffleHog missed. TruffleHog caught 449 Harpocrates missed.

TruffleHog's edge is live credential verification — it calls the AWS, GitHub, and Stripe APIs to confirm a key is still active. Harpocrates doesn't do that. If a key matches a known format and is still live, TruffleHog will catch it. Run both.

What Harpocrates catches that TruffleHog can't

1. Secrets under ambiguous variable names

TruffleHog requires the variable name to match a known pattern. Harpocrates uses ML to evaluate the token, the variable name, and the surrounding context together.

Code	TruffleHog	Harpocrates
client_secret = "eyJhbGci..."	❌	✅ JWT (0.97)
ENCRYPTION_KEY = "AKIAIOSFODNNabcd1234"	❌	✅ AWS key (0.90)
API_SECRET = "AKIAJOE7MTPSO7EXAM1E"	❌	✅ AWS key (0.80)
ACCESS_KEY = "eyJhbGciOiJSUzI1NiJ9..."	❌	✅ JWT (0.97)
my_token = "xoxb-12345-67890-abcdef..."	❌	✅ Slack token (0.94)

2. Secrets buried in comments

TruffleHog skips comment lines. Harpocrates scans all comment styles — #, //, /* */, <!--, -- — and flags any high-entropy token it finds there.

# old: DB_PASSWORD = "prod_hunter2_xxxSECRET"     ← TruffleHog: skipped. Harpocrates: caught.
// const apiKey = 'sk_live_NnyQ...';               ← TruffleHog: skipped. Harpocrates: caught.
TRACE_ID=$(git log) # AKIAMXWCUIXEKLN02TKZ         ← TruffleHog: sees git command. Harpocrates: sees the key in the comment.

3. Environment variable fallback leaks

A hardcoded fallback in os.getenv() is a secret that ships to production whenever the env var is unset. TruffleHog doesn't model this pattern. Harpocrates does.

SECRET_KEY = os.getenv("SECRET_KEY", "hardcoded_prod_value_xxx")   # ← caught
api_key = config("API_KEY", default="sk_live_realkey123456789")     # ← caught
const key = process.env.KEY ?? "literal_secret_value";             # ← caught

4. Dev/prod swap comments

Rotating from a test key to a production key and leaving the prod key in a comment is a common pattern. The comment leaks the live credential.

STRIPE_KEY = "sk_test_abc"  # PROD: sk_live_xyz_the_real_one    ← caught

5. AI-scaffolded code

AI coding assistants frequently hardcode credentials when scaffolding integrations. Harpocrates is trained on AI-generated code patterns specifically for this.

# Generated by Copilot:
client = OpenAI(api_key="sk-proj-xLMN3kJqPv...")    # ← caught
stripe.api_key = "sk_live_9QnKsF4jQ8M..."           # ← caught

6. Cross-language coverage

The same ML model covers Python, JavaScript/TypeScript, Go, Java, Ruby, YAML, Dockerfile, Terraform, and Helm values files. TruffleHog's pattern library varies by detector; Harpocrates's entropy+ML layer is language-agnostic.

Where TruffleHog has the edge

• Live verification — TruffleHog calls provider APIs to confirm a secret is active. Harpocrates never makes network calls.
• Format-anchored patterns — credentials that match a strict format (GitHub PAT prefix ghp_, Stripe's sk_live_) are caught with zero false positives by TruffleHog's regex. Harpocrates catches these too, but TruffleHog also confirms liveness.

Recommended setup: run Harpocrates as a pre-commit hook (catches the full range before anything reaches git) and TruffleHog in CI (confirms active credentials that made it through).

---

Install

pip install harpocrates
harpocrates scan .

With ML verification (recommended — 87% precision, 97% recall on held-out real-world set):

pip install "harpocrates[ml]"
harpocrates scan . --ml

With REST API server:

pip install "harpocrates[api]"
harpocrates serve

Everything at once:

pip install "harpocrates[all]"

---

Usage

# Scan a directory
harpocrates scan ./my_project

# Scan a single file
harpocrates scan config.env

# Output as JSON (pipe-friendly)
harpocrates scan ./my_project --json

# Enable ML verification to suppress false positives
harpocrates scan ./my_project --ml

# Only fail CI on high or critical findings
harpocrates scan ./my_project --fail-on high

# Ignore specific patterns
harpocrates scan ./my_project --ignore "*.test.js,fixtures/*"

Pre-commit hook

repos:
  - repo: https://github.com/Skipa776/Harpocrates
    rev: v0.2.0
    hooks:
      - id: harpocrates

Add to .pre-commit-config.yaml, then run pre-commit install. Harpocrates scans every staged file before each commit.

---

Configuration

harpocrates scan flags

Flag	Default	Description
--ml	off	Enable ML verification to reduce false positives
--ml-threshold FLOAT	0.19	ML confidence threshold 0.0–1.0. Lower = more recall, higher = more precision
--fail-on LEVEL	medium	Severity that triggers exit code 1: critical | high | medium | low | info | none
--json	off	Output results as JSON instead of a table
--show-secrets	off	Print full token values instead of redacted previews
--ignore TEXT	—	Comma-separated glob patterns to skip (e.g. "*.test.js,fixtures/*")
--max-size INTEGER	10	Maximum file size to scan, in MB
--recursive / --no-recursive	--recursive	Scan subdirectories recursively

Exit codes

Code	Meaning
0	No findings at or above --fail-on severity
1	One or more findings detected at or above --fail-on severity
2	Error (bad argument, unreadable file, etc.)

Other commands

harpocrates version          # Print version
harpocrates serve            # Start the REST API server (requires harpocrates[api])
harpocrates --help           # Full command list

---

How it scans

Harpocrates runs a three-phase pipeline on every line of every file:

1. Regex — deterministic patterns for known credential formats (AWS, GitHub, Stripe, private keys, and more). No ML required. High-confidence, zero false positives on well-formed keys.

2. Entropy analysis — Shannon entropy flags high-randomness tokens that don't match any known pattern. Catches credentials stored under ambiguous variable names (my_key, token, secret) that regex scanners miss entirely.

3. ML verification (opt-in via --ml) — a single-stage XGBoost classifier extracts 65 features from the token, its variable name, and the surrounding code context. It learns to distinguish api_secret = "AKIA..." (secret) from commit_sha = "a1b2c..." (Git SHA) without relying on the variable name alone. Inference runs via ONNX Runtime when available, with native XGBoost as fallback.

Ships pre-trained. No user training required.

The ML model is bundled with the package. pip install "harpocrates[ml]" is all you need.

---

MCP Server

Harpocrates ships an MCP (Model Context Protocol) server that exposes scan_text and scan_file to compatible agents via stdio transport.

Install:

pip install "harpocrates[mcp]"   # requires Python 3.10+

Run:

harpocrates-mcp                  # listens on stdio, speaks JSON-RPC

Tools exposed:

Tool	Arguments	Returns
scan_text	text: str, include_token: bool = False	List of finding dicts
scan_file	path: str, include_token: bool = False, max_bytes: int|None = None	List of finding dicts

Tokens are redacted by default (include_token=False). The MCP process has the same filesystem read scope as the user who launched it.

---

Version history

v0.3.0 — ML model retrained on 62k samples, comment scanning, violation classification

New ML capabilities (what the model can now detect that it couldn't before):

• Commented-out secrets — all comment styles now scanned: # (Python/YAML/Ruby), // (JS/TS/Go/Java), /* */ (C/CSS/Java), <!-- (HTML), -- (SQL/Lua). A credential commented out of active code is still in your git history forever.
• AI-scaffolded credentials — trained on LLM-generated code patterns where AI assistants hardcode credentials during integration scaffolding (OpenAI, Stripe, Twilio, Terraform, Helm, Dockerfile).
• Runtime generation negatives — the model now correctly ignores secrets.token_hex(32), crypto.randomBytes(32), uuid.uuid4().hex, SecureRandom.getInstanceStrong(). These are runtime generation calls, not stored secrets. v0.2 would flag them.
• Dev/prod swap patterns — KEY = "sk_test_x" # PROD: sk_live_y — the trailing comment leaks the production key.
• Duplicate-token lines — password = "password123", token = "token-prod-abc...". The ML model now handles lines where the variable name appears as a substring of the value, which confused earlier models.
• Env-var fallback leaks — expanded coverage across Django (config(..., default=)), Pydantic Settings, Spring Boot @Value, Rails credentials.dig, Go, Node.js (??), TypeScript/Zod, Helm | default.
• Violation classification — every finding now includes a category field (password, api_token, connection_string, jwt, private_key, crypto_key, oauth_secret, session_token, webhook_url, generic_secret) and a category_reason string explaining which heuristic fired. This tells you what to rotate, not just that something was found.

Model metrics (v5 corpus, 62k training samples):

Set	Recall	Precision	F1	AUC-ROC
Synthetic test (internal)	99.15%	94.61%	0.968	0.9957
Held-out OOD (TruffleHog-validated)	97.32%	87.14%	0.920	0.9823

Threshold low (recall gate): 0.15 — anything above this is flagged for review. Threshold high (precision gate): 0.85 — anything above this is reported as a confirmed secret.

---

v0.2.0 — Three-phase pipeline, MCP server, pre-commit hook

ML capabilities added:

• XGBoost + ONNX Runtime — single-stage classifier with 65 features. Replaces the v0.1 heuristic confidence scores with a calibrated probability (Platt-scaled).
• 65-feature context model — captures token entropy, variable name semantics (var_contains_secret, var_ngram_secret_score), file type risk (file_is_config, file_extension_risk), surrounding context (context_has_function_def), and value structure (is_known_hash_length, hex_adjacent_assignment_pattern, has_padding, is_test_token, line_position_ratio).
• Dual-threshold routing — findings are routed to SAFE (below threshold_low), REVIEW (between thresholds), or SECRET (above threshold_high). The review zone is ~5% of lines on average.
• Entropy candidates surfaced — high-entropy tokens under any variable name now reach the ML stage, not just those matching known patterns.
• MCP server — harpocrates-mcp exposes scan_text and scan_file as JSON-RPC tools for agent-to-agent scanning (Claude, Cursor, Codeium, etc.).
• TokenMatch infrastructure — internal span tracking for exact token position within a line, required for the v0.3 feature-set retrain.

Regex signatures added in v0.2:

Credential	Pattern
OpenAI API key	sk-... / sk-proj-...
Anthropic Claude key	sk-ant-api03-...
GCP API key	AIza...
NPM token	npm_...
PyPI token	pypi-...
HashiCorp Vault token	hvs....

---

v0.1.0 — Initial release ("the wedge")

What shipped:

• Three-phase detection pipeline: Regex → Shannon Entropy → ML confidence scoring.
• 10 regex signatures: AWS Access Key ID, GitHub PAT, Slack token, Stripe key, private key PEM header, Slack webhook, Discord webhook, SendGrid key, Twilio SID, Databricks token.
• Pre-commit hook — scans every staged file before commit. Handles binary files and UnicodeDecodeError without crashing.
• CLI: harpocrates scan, --json, --fail-on, --ml-threshold, --show-secrets, --ignore.
• Packaged as py3-none-any.whl — no native extensions, installs on any platform without compilation.

---

Contributing

False negatives are the highest-priority reports. If Harpocrates missed a real secret, open an issue with the false-negative label and include the variable name pattern and secret type. This is the most valuable feedback you can give.

For bugs, feature requests, and false positives, open an issue at github.com/Skipa776/Harpocrates/issues.

---

License

MIT — see LICENSE.