OWASP DeepSecrets — a better tool for secrets scanning
What is it? Another token-wasting CLI proxy to an AI API?
Absolutely not!
In our LLM-hype era, DeepSecrets still runs entirely on your machine — giving you great results offline, securely, and for free.
So why yet another tool?
Most existing scanners don't actually "understand" code. Instead, they just parse texts and have bad coverage.
DeepSecrets bridges the gap between classic regex scanners and full-scale commercial SAST tools. It extends the classic regex-based scanning strategy by heavily relying on semantic code analysis, dangerous variable detection, and context-aware entropy analysis. Secret candidates come from what the code says, its names, assignments and calls, not from text alone. DeepSecrets reads code with lexers for about 600 languages and formats (Pygments), and finds variables with detection rules for 14 languages and a generic rule for the rest.
DeepSecrets also introduces a new way to find credentials with zero knowledge: the HashedSecret Engine. Just provide the hashed values of your known production secrets, and the tool will find them exposed in plain text within your code.
Performance & Benchmarks (SecretBench)
DeepSecrets 2.3 was evaluated (November 2026) on the SecretBench benchmark: every file scanned, every labelled row counted.
- 93% Recall and 91% Precision (2.2: 93% and 86%)
- 1.6% False Positive Rate on SecretBench scope (2.2: 2.7%)
- About 2,000 more secrets that SecretBench does not label, among its findings outside the labels
Counted the way the June 2026 evaluation counted, on its own set of rows, 2.3 has 94% recall and 85% precision (2.0 in June 2026: 93% and 67%).
Reproduce these numbers
The figures come from dsbench, an open-source benchmark harness that runs a scanner as a black box over a labelled corpus and counts what it reports. With SecretBench (available from its authors on request) and dsbench, anyone can reproduce the per-row figures above and the comparison below:
dsbench prepare # load SecretBench's labels and the corpus listing
dsbench run --tool-ref v2.3.0 # DeepSecrets 2.3.0 from its tag, in a venv from its own poetry.lock
dsbench tools fetch # the pinned gitleaks and Betterleaks releases, checked by SHA-256
dsbench run --tool gitleaks
dsbench run --tool betterleaks
dsbench compare RUN RUN ... # side by side, with 95% intervals
Every run scans every file, with each tool's path exclusions turned off. The June-way figures also need the June 2026 evaluation's table of rows, and the count of secrets beyond the labels needs our judged verdicts; neither is published, because both point at leaked credentials.
(You can read the full under-the-hood story and benchmark breakdown in my HackerNoon article here)
Quick Start Guide
Installation
From Github via pip
$ pip install git+https://github.com/ntoskernel/deepsecrets.git
From PyPi
$ pip install deepsecrets
DeepSecrets needs Python 3.11 or later.
Scanning
The easiest way to run a scan:
$ deepsecrets --target-dir /path/to/your/code --outformat dojo-sarif --outfile report.json
This will run a scan against /path/to/your/code using the default configuration:
- Regex using the built-in ruleset
- Semantic checks (variable detection, entropy checks)
A report in SARIF format (compatible with DefectDojo and GitHub Security) will be saved to report.json.
Fine-Tuning
The --help command is always ready to guide you, but here are the key flags you can use to tailor the scan to your environment:
--regex-rules /path/to/rules.json: Supply your own custom regex ruleset.--hashed-values /path/to/hashes.json: Provide a list of pre-hashed known production secrets to search for them securely.--excluded-paths /path/to/exclusions.json: Override or extend the default paths ignored during scanning.--deep-max-size BYTES: Files larger than this (default 250,000 bytes) get the fast analysis: the regex rules and the variable search, without the language lexer. Big files are mostly generated or minified, the lexer costs most of the scan time on them and rarely finds more there.0gives every file the full analysis.--skip-bundles: Skip minified JavaScript, source maps and bundles (*.min.js,*.map,*.bundle.js) entirely, on top of the other path exclusions.--confidence-level {all,low,medium,high,very-high}: The lowest confidence to report (defaultlow, every finding).medium,highandvery-highkeep confidence 5, 6 and 9 or more, the findings of the SARIF tiers of the same names and above;allalso reports the regex matches the scanner judged not to be secrets (placeholders, examples), with an-INFOrule id.--min-confidence N: Report findings of confidenceNor more,Nfrom 0 to 10: any cut, where--confidence-leveloffers the floors of the tiers. Like every level butall, it leaves out the regex matches judged not to be secrets. Use one flag or the other, not both.--exclude-tags TAG [TAG ...]: Leave out the findings that carry any of these tags. A tag says what sort of secret a finding is, where its confidence says how likely it is one:weak-password(a weak value under a password-like name),publishable(a key made to ship to clients: reCAPTCHA and other site keys, Stripepk_keys, Mapboxpk.tokens, player, analytics and maps keys),test-context(a test, spec, fixture, mock, example or sample path, relative to the target directory),recorded-traffic(VCR cassettes, session recordings, HAR files),identifier(an AWS id of a user, role or policy, which grants nothing) andweb-page(an HTML page). Every finding is still reported by default; e.g.--exclude-tags test-contextwhen you hunt live secrets.--report-diagnostics: Add every file found under the target to the SARIF report, with its status, scan time, skip reason and analysis depth.--regex-candidate-scoring-rules /path/to/rules.json: Supply your own rules for judging regex matches before they are reported. Give the flag no value to report regex matches unjudged.--multiprocessing-context {forkserver,spawn,fork}: How worker processes start. The default isforkserverwhere the platform has it (Linux, macOS): workers share the scanner already loaded in a server process. Windows usesspawn. All three give the same findings.--disable-masking: Keep potential secrets unmasked in the output report (see caution below).--ci/--no-ci: Turn CI mode on or off. CI mode prints a plain progress line every 30 seconds instead of the live progress display. It turns on by itself inside CI services (GitHub Actions, GitLab CI, Jenkins and others) and whenever the output is not a terminal.
Github Actions Integration
eq. .github/workflows/deepsecrets.yml
name: DeepSecrets Scan
on: [push, pull_request]
jobs:
scan:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.14'
- name: Install DeepSecrets
run: pip install deepsecrets
- name: Run Scan
run: deepsecrets --target-dir . --outformat dojo-sarif --outfile report.sarif
continue-on-error: true
- name: Upload SARIF report
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: report.sarif
Masking Secrets in Reports
As of v1.3.0, potential secrets are automatically masked inside reports to protect your pipeline artifacts. Turn this off via the --disable-masking flag if necessary.
Masking doesn't break the deduplication logic of downstream platforms (like Github), as the partialFingerprints section in the report is correctly populated based on the raw data.
SARIF Reports & Dynamic Confidence
Every finding gets a confidence score. However, different security platforms parse SARIF metrics differently. To ensure compatibility across modern ASPM dashboards, DeepSecrets does the following:
-
Virtual Subrules (
rules[]): Dynamically generates rules by confidence tier, fromS105-LOW(confidence 0 to 4) through-MEDIUM(5) and-HIGH(6 to 8) toS105-VERY-HIGH(9 and 10), and-INFOfor the rejected matches--confidence-level allreports. This forces GitHub Security and DefectDojo to map semantic precision variance properly without breaking native parsers. -
Deterministic Result Level: The tool always explicitly sets
level: errorin theresults[]model. This acts as a universal fallback for CI/CD pipelines and older SAST parsers, ensuring that exposed secrets reliably break builds or block Pull Requests regardless of individual rule interpretations. -
Tags: Each result carries its tags, if it has any, in SARIF's standard
properties.tagsarray (weak-password,publishable,test-context,recorded-traffic,identifier,web-page). Rule ids, severity and fingerprints do not depend on them. -
Contextual Messages: Injects the raw numeric confidence score natively into
result.message.textso security analysts see it immediately on their dashboards.
Building rulesets
Regex
The built-in ruleset for regex checks is located in /deepsecrets/rules/regexes.json. You're free to follow the format and create a custom ruleset.
HashedSecret (Zero-Knowledge Scanning)
Example ruleset for hashed checks is located in /tests/fixtures/hashed_secrets.json. You're free to follow the format and create a custom ruleset.
HashedSecret Ruleset Example
To look for known production secrets without exposing them in plaintext inside your repository, provide a JSON containing their hashes:
[
{
"name": "KNOWN-PROD-DATABASE-PASSWORD",
"hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
"length": 12,
"algorithm": "sha1"
}
]
Run with --hashed-values /path/to/hashes.json. DeepSecrets will automatically hash string candidates on the fly during its lexing stage to match them.
Contacts
- Nikolai Khechumov (@ntoskernel) — creator and maintainer
FAQ
Pff, is it still regex-based?
Yes and no. Of course, it uses regexes to find typed secrets like any other tool. But language understanding (the lexing stage) and variable detection also use regexes under the hood. Regex is an instrument, not the problem. The problem is applying regex blindly without semantic context.
But what about Semgrep Secrets? Looks like you're cloning their thing.
DeepSecrets was originally released in April 2023 — six months before Semgrep Secrets launched. We share similar principles, but DeepSecrets is 100% free/open-source and leverages a significantly broader multi-language tracking surface.
DeepSecrets vs. Other Scanners
While other tools scan only what they know, DeepSecrets leverages lexers. This allows it to surface hidden, dangerous credentials in rare configuration formats and custom code blocks that benchmarks may not have datasets for.
Tool comparison based on SecretBench Results
| Feature / Capability | DeepSecrets 2.3 | Gitleaks 8.30 | Betterleaks 1.8 | TruffleHog | Semgrep Secrets |
|---|---|---|---|---|---|
| SecretBench Accuracy | 93% Recall 91% Precision |
85% Recall 70% Precision |
85% Recall 61% Precision |
Not re-measured | Not Evaluated |
| Price & Licensing | Free / Open-Source | Free / Open-Source | Free / Open-Source | Free / Open-Source | Commercial / Paid |
| Analysis Type | Semantic / Regex | Flat-text Regex / Entropy | Flat-text Regex / Entropy | Flat-text Regex / Entropy | Semantic |
| Language Support | ~600 lexed (variables in 14) |
Context-agnostic (Text) | Context-agnostic (Text) | Context-agnostic (Text) | Limited subset |
| Pre-hashed Validation | Yes (via Hashed Engine) | No | No | No | No |
| Context-Aware Entropy | Yes (Assigned values) | No (Entire file text) | No (Entire file text) | No (Entire file text) | Yes |
| Advanced SARIF Output | Yes (Confidence tiers and tags) | Basic | Basic | Basic | Yes |
SecretBench, every file scanned, every labelled row counted, with dsbench (see Reproduce these numbers). DeepSecrets 2.3, Gitleaks 8.30.1 and Betterleaks 1.8.1 measured on the same harness in October and November 2026, Gitleaks and Betterleaks with their default rules and their global path allowlist removed so that every file is scanned, as for DeepSecrets. TruffleHog was measured in June 2026 by another method (52% recall, 6% precision) and not since.
Why don't you build true abstract syntax trees? It's academically more correct!
DeepSecrets tries to keep a balance between complexity and effectiveness. Building a true AST across hundreds of languages is incredibly complex and simply overkill for the secrets detection. The tool follows the generic SAST approach to code analysis but optimizes the AST stage for maximum speed and width.
I'd like to build my own semantic rules. How do I do that?
Semantic rules are now effectively "variable evaluation rules". You can find them here.
I still have a question
Feel free to contact the developer directly using the emails listed in pyproject.toml
Contributing & Core Concepts
Under the hood
There are several core concepts:
FileTokenizerTokenEngineFindingScanMode
File
Just a pythonic representation of a file with all needed methods for management.
Tokenizer
Breaks the content of a file into pieces - Tokens - by its logic. There are four types of tokenizers available:
FullContentTokenizer: treats all content as a single token. Useful for regex-based search.PerWordTokenizer: breaks given content by words and line breaks.LexerTokenizer: uses language-specific smarts to break code into semantically correct pieces with additional context for each token.CheapVarDetectorTokenizer: uses tight regexes to cover limitations of semantic variable detection.
Token
A string with additional information about its semantic role, corresponding file, and location inside it.
Engine
A component performing secrets search for a single token by its own logic. Returns a set of Findings. There are three engines available:
RegexEngine: checks tokens' values through a special rulesetSemanticEngine: checks tokens produced by the LexerTokenizer using additional context - variable names and valuesHashedSecretEngine: checks tokens' values by hashing them and trying to find coinciding hashes inside a special ruleset
Finding
This is a data structure representing a problem detected inside code. Features information about the precise location inside a file and a rule that found it.
ScanMode
This component is responsible for the scan process.
- Defines the scope of analysis for a given work directory respecting exceptions
- Allows declaring a
PerFileAnalyzer- the method called against each file, returning a list of findings. The primary usage is to initialize necessary engines, tokenizers, and rulesets. - Runs the scan: a multiprocessing pool analyzes every file in parallel.
- Prepares results for output and outputs them.
The current implementation has a CliScanMode built by the user-provided config through the cli args.
Local development
The project is supposed to be developed using VSCode and 'Remote containers' feature.
Steps:
- Clone the repository
- Open the cloned folder with VSCode
- Select "Reopen in Container" when prompted
- Wait for the automated environment build to complete. You are ready to develop.
History and license
DeepSecrets was created by Nikolai Khechumov at Avito in 2022 and open-sourced in April 2023 at avito-tech/deepsecrets. It has been an OWASP project since 2023 and has been developed in this repository since November 2023.
From version 2.3.0 DeepSecrets is licensed under the Apache License 2.0. Code from the original release and contributions made before 2.3.0 remain available under the MIT License, whose text is at the end of LICENSE; NOTICE says which is which. Releases up to and including 2.2.0 were published under the MIT License.
Metadata
Release files for deepsecrets 2.3.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| deepsecrets-2.3.0.tar.gz | 666.5 kB | With provenance | Details for deepsecrets-2.3.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| deepsecrets-2.3.0-py3-none-any.whl | Python 3 | none | any | Details for deepsecrets-2.3.0-py3-none-any.whl |
Total release size: 1.4 MB
Release files / deepsecrets-2.3.0.tar.gz
| Download URL | deepsecrets-2.3.0.tar.gz |
|---|---|
| Size | 666.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
9b5fe7419678cda8be2849ec4796a81a4a22bc37da0cc7f24bd866afe32e65a4
|
|
BLAKE2b-256 checksum How to use checksums |
02ad2fcf558cce432a8ca0b47fd7372d74de5280bb199f1e89cb3a0ea201c00c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 10, 2026.
Transparency logRelease files / deepsecrets-2.3.0-py3-none-any.whl
| Download URL | deepsecrets-2.3.0-py3-none-any.whl |
|---|---|
| Size | 695.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
8ef4ec072d9c784e2ea22d20d8aa15a592fe0e0477354847002d84c63e9ad403
|
|
BLAKE2b-256 checksum How to use checksums |
816bbe8dcc85484b3552e8d35d197d2bef3476e569622e91e5076b3aa58a0128
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 10, 2026.
Transparency logSecurity for deepsecrets 2.3.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for deepsecrets-2.3.0.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 10, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.