Skip to main content

OWASP DeepSecrets — a better tool for secrets scanning

Tests Status

What is it? Another token-wasting CLI proxy to an AI API?

Absolutely not!

In our LLM-hype era, DeepSecrets still runs entirely on your machine — giving you great results offline, securely, and for free.

So why yet another tool?

Most existing scanners don't actually "understand" code. Instead, they just parse texts and have bad coverage.

DeepSecrets bridges the gap between classic regex scanners and full-scale commercial SAST tools. It extends the classic regex-based scanning strategy by heavily relying on semantic code analysis, dangerous variable detection, and context-aware entropy analysis. Secret candidates come from what the code says, its names, assignments and calls, not from text alone. DeepSecrets reads code with lexers for about 600 languages and formats (Pygments), and finds variables with detection rules for 14 languages and a generic rule for the rest.

DeepSecrets also introduces a new way to find credentials with zero knowledge: the HashedSecret Engine. Just provide the hashed values of your known production secrets, and the tool will find them exposed in plain text within your code.

Performance & Benchmarks (SecretBench)

DeepSecrets 2.3 was evaluated (November 2026) on the SecretBench benchmark: every file scanned, every labelled row counted.

  • 93% Recall and 91% Precision (2.2: 93% and 86%)
  • 1.6% False Positive Rate on SecretBench scope (2.2: 2.7%)
  • About 2,000 more secrets that SecretBench does not label, among its findings outside the labels

Counted the way the June 2026 evaluation counted, on its own set of rows, 2.3 has 94% recall and 85% precision (2.0 in June 2026: 93% and 67%).

Reproduce these numbers

The figures come from dsbench, an open-source benchmark harness that runs a scanner as a black box over a labelled corpus and counts what it reports. With SecretBench (available from its authors on request) and dsbench, anyone can reproduce the per-row figures above and the comparison below:

dsbench prepare                        # load SecretBench's labels and the corpus listing
dsbench run --tool-ref v2.3.0          # DeepSecrets 2.3.0 from its tag, in a venv from its own poetry.lock
dsbench tools fetch                    # the pinned gitleaks and Betterleaks releases, checked by SHA-256
dsbench run --tool gitleaks
dsbench run --tool betterleaks
dsbench compare RUN RUN ...            # side by side, with 95% intervals

Every run scans every file, with each tool's path exclusions turned off. The June-way figures also need the June 2026 evaluation's table of rows, and the count of secrets beyond the labels needs our judged verdicts; neither is published, because both point at leaked credentials.

(You can read the full under-the-hood story and benchmark breakdown in my HackerNoon article here)

Quick Start Guide

Installation

From Github via pip

$ pip install git+https://github.com/ntoskernel/deepsecrets.git

From PyPi

$ pip install deepsecrets

DeepSecrets needs Python 3.11 or later.

Scanning

The easiest way to run a scan:

$ deepsecrets --target-dir /path/to/your/code --outformat dojo-sarif --outfile report.json

This will run a scan against /path/to/your/code using the default configuration:

  • Regex using the built-in ruleset
  • Semantic checks (variable detection, entropy checks)

A report in SARIF format (compatible with DefectDojo and GitHub Security) will be saved to report.json.

Fine-Tuning

The --help command is always ready to guide you, but here are the key flags you can use to tailor the scan to your environment:

  • --regex-rules /path/to/rules.json: Supply your own custom regex ruleset.
  • --hashed-values /path/to/hashes.json: Provide a list of pre-hashed known production secrets to search for them securely.
  • --excluded-paths /path/to/exclusions.json: Override or extend the default paths ignored during scanning.
  • --deep-max-size BYTES: Files larger than this (default 250,000 bytes) get the fast analysis: the regex rules and the variable search, without the language lexer. Big files are mostly generated or minified, the lexer costs most of the scan time on them and rarely finds more there. 0 gives every file the full analysis.
  • --skip-bundles: Skip minified JavaScript, source maps and bundles (*.min.js, *.map, *.bundle.js) entirely, on top of the other path exclusions.
  • --confidence-level {all,low,medium,high,very-high}: The lowest confidence to report (default low, every finding). medium, high and very-high keep confidence 5, 6 and 9 or more, the findings of the SARIF tiers of the same names and above; all also reports the regex matches the scanner judged not to be secrets (placeholders, examples), with an -INFO rule id.
  • --min-confidence N: Report findings of confidence N or more, N from 0 to 10: any cut, where --confidence-level offers the floors of the tiers. Like every level but all, it leaves out the regex matches judged not to be secrets. Use one flag or the other, not both.
  • --exclude-tags TAG [TAG ...]: Leave out the findings that carry any of these tags. A tag says what sort of secret a finding is, where its confidence says how likely it is one: weak-password (a weak value under a password-like name), publishable (a key made to ship to clients: reCAPTCHA and other site keys, Stripe pk_ keys, Mapbox pk. tokens, player, analytics and maps keys), test-context (a test, spec, fixture, mock, example or sample path, relative to the target directory), recorded-traffic (VCR cassettes, session recordings, HAR files), identifier (an AWS id of a user, role or policy, which grants nothing) and web-page (an HTML page). Every finding is still reported by default; e.g. --exclude-tags test-context when you hunt live secrets.
  • --report-diagnostics: Add every file found under the target to the SARIF report, with its status, scan time, skip reason and analysis depth.
  • --regex-candidate-scoring-rules /path/to/rules.json: Supply your own rules for judging regex matches before they are reported. Give the flag no value to report regex matches unjudged.
  • --multiprocessing-context {forkserver,spawn,fork}: How worker processes start. The default is forkserver where the platform has it (Linux, macOS): workers share the scanner already loaded in a server process. Windows uses spawn. All three give the same findings.
  • --disable-masking: Keep potential secrets unmasked in the output report (see caution below).
  • --ci / --no-ci: Turn CI mode on or off. CI mode prints a plain progress line every 30 seconds instead of the live progress display. It turns on by itself inside CI services (GitHub Actions, GitLab CI, Jenkins and others) and whenever the output is not a terminal.

Github Actions Integration

eq. .github/workflows/deepsecrets.yml

name: DeepSecrets Scan
on: [push, pull_request]

jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.14'

      - name: Install DeepSecrets
        run: pip install deepsecrets

      - name: Run Scan
        run: deepsecrets --target-dir . --outformat dojo-sarif --outfile report.sarif
        continue-on-error: true

      - name: Upload SARIF report
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: report.sarif

Masking Secrets in Reports

As of v1.3.0, potential secrets are automatically masked inside reports to protect your pipeline artifacts. Turn this off via the --disable-masking flag if necessary.

Masking doesn't break the deduplication logic of downstream platforms (like Github), as the partialFingerprints section in the report is correctly populated based on the raw data.

SARIF Reports & Dynamic Confidence

Every finding gets a confidence score. However, different security platforms parse SARIF metrics differently. To ensure compatibility across modern ASPM dashboards, DeepSecrets does the following:

  • Virtual Subrules (rules[]): Dynamically generates rules by confidence tier, from S105-LOW (confidence 0 to 4) through -MEDIUM (5) and -HIGH (6 to 8) to S105-VERY-HIGH (9 and 10), and -INFO for the rejected matches --confidence-level all reports. This forces GitHub Security and DefectDojo to map semantic precision variance properly without breaking native parsers.

  • Deterministic Result Level: The tool always explicitly sets level: error in the results[] model. This acts as a universal fallback for CI/CD pipelines and older SAST parsers, ensuring that exposed secrets reliably break builds or block Pull Requests regardless of individual rule interpretations.

  • Tags: Each result carries its tags, if it has any, in SARIF's standard properties.tags array (weak-password, publishable, test-context, recorded-traffic, identifier, web-page). Rule ids, severity and fingerprints do not depend on them.

  • Contextual Messages: Injects the raw numeric confidence score natively into result.message.text so security analysts see it immediately on their dashboards.

Building rulesets

Regex

The built-in ruleset for regex checks is located in /deepsecrets/rules/regexes.json. You're free to follow the format and create a custom ruleset.

HashedSecret (Zero-Knowledge Scanning)

Example ruleset for hashed checks is located in /tests/fixtures/hashed_secrets.json. You're free to follow the format and create a custom ruleset.

HashedSecret Ruleset Example

To look for known production secrets without exposing them in plaintext inside your repository, provide a JSON containing their hashes:

[
  {
    "name": "KNOWN-PROD-DATABASE-PASSWORD",
    "hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
    "length": 12,
    "algorithm": "sha1"
  }
]

Run with --hashed-values /path/to/hashes.json. DeepSecrets will automatically hash string candidates on the fly during its lexing stage to match them.

Contacts

  • Nikolai Khechumov (@ntoskernel) — creator and maintainer

FAQ

Pff, is it still regex-based?

Yes and no. Of course, it uses regexes to find typed secrets like any other tool. But language understanding (the lexing stage) and variable detection also use regexes under the hood. Regex is an instrument, not the problem. The problem is applying regex blindly without semantic context.

But what about Semgrep Secrets? Looks like you're cloning their thing.

DeepSecrets was originally released in April 2023 — six months before Semgrep Secrets launched. We share similar principles, but DeepSecrets is 100% free/open-source and leverages a significantly broader multi-language tracking surface.

DeepSecrets vs. Other Scanners

While other tools scan only what they know, DeepSecrets leverages lexers. This allows it to surface hidden, dangerous credentials in rare configuration formats and custom code blocks that benchmarks may not have datasets for.

Tool comparison based on SecretBench Results

Feature / Capability DeepSecrets 2.3 Gitleaks 8.30 Betterleaks 1.8 TruffleHog Semgrep Secrets
SecretBench Accuracy 93% Recall
91% Precision
85% Recall
70% Precision
85% Recall
61% Precision
Not re-measured Not Evaluated
Price & Licensing Free / Open-Source Free / Open-Source Free / Open-Source Free / Open-Source Commercial / Paid
Analysis Type Semantic / Regex Flat-text Regex / Entropy Flat-text Regex / Entropy Flat-text Regex / Entropy Semantic
Language Support ~600 lexed
(variables in 14)
Context-agnostic (Text) Context-agnostic (Text) Context-agnostic (Text) Limited subset
Pre-hashed Validation Yes (via Hashed Engine) No No No No
Context-Aware Entropy Yes (Assigned values) No (Entire file text) No (Entire file text) No (Entire file text) Yes
Advanced SARIF Output Yes (Confidence tiers and tags) Basic Basic Basic Yes

SecretBench, every file scanned, every labelled row counted, with dsbench (see Reproduce these numbers). DeepSecrets 2.3, Gitleaks 8.30.1 and Betterleaks 1.8.1 measured on the same harness in October and November 2026, Gitleaks and Betterleaks with their default rules and their global path allowlist removed so that every file is scanned, as for DeepSecrets. TruffleHog was measured in June 2026 by another method (52% recall, 6% precision) and not since.

Why don't you build true abstract syntax trees? It's academically more correct!

DeepSecrets tries to keep a balance between complexity and effectiveness. Building a true AST across hundreds of languages is incredibly complex and simply overkill for the secrets detection. The tool follows the generic SAST approach to code analysis but optimizes the AST stage for maximum speed and width.

I'd like to build my own semantic rules. How do I do that?

Semantic rules are now effectively "variable evaluation rules". You can find them here.

I still have a question

Feel free to contact the developer directly using the emails listed in pyproject.toml

Contributing & Core Concepts

Under the hood

There are several core concepts:

  • File
  • Tokenizer
  • Token
  • Engine
  • Finding
  • ScanMode

File

Just a pythonic representation of a file with all needed methods for management.

Tokenizer

Breaks the content of a file into pieces - Tokens - by its logic. There are four types of tokenizers available:

  • FullContentTokenizer: treats all content as a single token. Useful for regex-based search.
  • PerWordTokenizer: breaks given content by words and line breaks.
  • LexerTokenizer: uses language-specific smarts to break code into semantically correct pieces with additional context for each token.
  • CheapVarDetectorTokenizer: uses tight regexes to cover limitations of semantic variable detection.

Token

A string with additional information about its semantic role, corresponding file, and location inside it.

Engine

A component performing secrets search for a single token by its own logic. Returns a set of Findings. There are three engines available:

  • RegexEngine: checks tokens' values through a special ruleset
  • SemanticEngine: checks tokens produced by the LexerTokenizer using additional context - variable names and values
  • HashedSecretEngine: checks tokens' values by hashing them and trying to find coinciding hashes inside a special ruleset

Finding

This is a data structure representing a problem detected inside code. Features information about the precise location inside a file and a rule that found it.

ScanMode

This component is responsible for the scan process.

  • Defines the scope of analysis for a given work directory respecting exceptions
  • Allows declaring a PerFileAnalyzer - the method called against each file, returning a list of findings. The primary usage is to initialize necessary engines, tokenizers, and rulesets.
  • Runs the scan: a multiprocessing pool analyzes every file in parallel.
  • Prepares results for output and outputs them.

The current implementation has a CliScanMode built by the user-provided config through the cli args.

Local development

The project is supposed to be developed using VSCode and 'Remote containers' feature.

Steps:

  1. Clone the repository
  2. Open the cloned folder with VSCode
  3. Select "Reopen in Container" when prompted
  4. Wait for the automated environment build to complete. You are ready to develop.

History and license

DeepSecrets was created by Nikolai Khechumov at Avito in 2022 and open-sourced in April 2023 at avito-tech/deepsecrets. It has been an OWASP project since 2023 and has been developed in this repository since November 2023.

From version 2.3.0 DeepSecrets is licensed under the Apache License 2.0. Code from the original release and contributions made before 2.3.0 remain available under the MIT License, whose text is at the end of LICENSE; NOTICE says which is which. Releases up to and including 2.2.0 were published under the MIT License.

Metadata

Release files for deepsecrets 2.3.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for deepsecrets 2.3.0
File Size Uploaded Provenance Actions
deepsecrets-2.3.0.tar.gz 666.5 kB With provenance Details for deepsecrets-2.3.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for deepsecrets 2.3.0
File Interpreter ABI Platform Actions
deepsecrets-2.3.0-py3-none-any.whl Python 3 none any Details for deepsecrets-2.3.0-py3-none-any.whl

Total release size: 1.4 MB

Release files / deepsecrets-2.3.0.tar.gz

Download URL deepsecrets-2.3.0.tar.gz
Size 666.5 kB
Tags Source
SHA-256 checksum
How to use checksums
9b5fe7419678cda8be2849ec4796a81a4a22bc37da0cc7f24bd866afe32e65a4
BLAKE2b-256 checksum
How to use checksums
02ad2fcf558cce432a8ca0b47fd7372d74de5280bb199f1e89cb3a0ea201c00c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 10, 2026.

Transparency log

Release files / deepsecrets-2.3.0-py3-none-any.whl

Download URL deepsecrets-2.3.0-py3-none-any.whl
Size 695.1 kB
Tags Python 3
SHA-256 checksum
How to use checksums
8ef4ec072d9c784e2ea22d20d8aa15a592fe0e0477354847002d84c63e9ad403
BLAKE2b-256 checksum
How to use checksums
816bbe8dcc85484b3552e8d35d197d2bef3476e569622e91e5076b3aa58a0128
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 10, 2026.

Transparency log

Security for deepsecrets 2.3.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for deepsecrets-2.3.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 10, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page