credsweeper 1.14.7

Credential Sweeper

CredSweeper

• CredSweeper
  • Introduction
  • How To Use
    • Main Requirements
    • Installation
    • Run
    • Config
  • Develop
    • Tests
    • Benchmark
  • Overall Architecture
  • Retrain Model
  • License
  • How to Get Involved
    • Project Roles
      • Contributor
      • Maintainer
  • How to Contact

Introduction

CredSweeper is an advanced credential detection tool designed to identify exposed credentials such as passwords, API keys, tokens, and other sensitive information across source code, configuration files, documents, and binary assets. CredSweeper scans regular files, embedded data in containers, and files added in Git commits. The tool combines pattern-based detection, machine learning–based validation, and deep file inspection to deliver comprehensive and accurate security scanning for modern codebases and repositories.

Key Capabilities:

• Credential detection in source code, configuration files, documents, and archives
• False positive reduction using algorithmic filters and machine learning
• Scanning of compressed files, documents, and binary formats
• Git repository analysis and diff scanning

Full documentation can be found here: https://credsweeper.readthedocs.io/

How To Use

Main Requirements

• Python 3.10, 3.11, 3.12

Installation

Details here.

pip install credsweeper

Run

How to use.

Run CredSweeper:

python -m credsweeper --path tests/samples/password.gradle --save-json output.json

JSON Output

[
    {
        "rule": "Password",
        "severity": "high",
        "confidence": "moderate",
        "ml_probability": 0.993,
        "line_data_list": [
            {
                "line": "password = \"cackle!\"",
                "line_num": 1,
                "path": "./tests/samples/password.gradle",
                "info": "",
                "variable": "password",
                "variable_start": 0,
                "variable_end": 8,
                "value": "cackle!",
                "value_start": 12,
                "value_end": 19,
                "entropy": 2.52164
            }
        ]
    }
]

Config

credsweeper/secret/config.json - Configuration file for pre-processing of CredSweeper. For more details please check here.

You can set the pattern, extension and path you want to exclude from scanning as below.

{
    "exclude": {
        "pattern": [
            "AKIA[0-9A-Z]{9}EXAMPLE",
            ...
        ],
        "extension": [
            "gif",
            "jpg",
            ...
        ],
        "path": [
            "/.git/",
            "/openssl/",
            ...
        ]
    },
    ...
}

And you can also set source_ext, source_quote_ext, find_by_ext_list, check_for_literals, line_data_output, and candidate_output as below.

• source_ext: List of extensions for scanning categorized as source files.
• source_quote_ext: List of extensions for scanning categorized as source files that use quotes.
• find_by_ext_list: List of extensions to detect only extensions.
• check_for_literals: Bool value for whether to check line has string literal declaration or not.
• line_data_output: List of attributes of line_data for output.
• candidate_output: List of attributes of candidate for output.

{
    ...
    "source_ext": [
        ".py",
        ".cpp",
        ...
    ],
    "source_quote_ext": [
        ".py",
        ".cpp",
        ...
    ],
    "find_by_ext_list": [
        ".pem",
        ".cer",
        ...
    ],
    "check_for_literals": true,
    "line_data_output": [
        "line",
        "line_num",
        ...
    ],
    "candidate_output": [
        "rule",
        "severity",
        ...
    ]
}

credsweeper/rules/config.yaml - Configuration file for setting Rule. For more details please check here.

- name: Credential
  severity: medium
  confidence: moderate
  type: keyword
  values:
    - credential
  filter_type: GeneralKeyword
  use_ml: true
  min_line_len: 18
  required_substrings:
    - credential
  target:
    - code

Develop

Tests

Run all tests with random order:

python -m pytest --cov=credsweeper --cov-report=term-missing --random-order --random-order-bucket=global -s tests/

Benchmark

We have a dataset for testing credential scanners called CredData. If you want to test CredSweeper with this dataset please check here.

Overall Architecture

To check overall architecture of CredSweeper please check here.

Retrain Model

If you want to check how model was trained or retrain it on your own data, please refer to the experiment folder

License

The CredSweeper is an Open Source project released under the terms of MIT License.

How to Get Involved

In addition to developing under an Open Source license, the project follows an Open Source Development approach, welcoming everyone to participate, contribute, and engage with each other through the project.

Project Roles

The project recognizes the following formal roles: Contributor and Maintainer. Informally, the community may organize itself and grant additional rights and responsibilities to the necessary people to achieve its goals.

Contributor

A Contributor is anyone who wishes to contribute to the project, at any level. Contributors are granted the following rights to:

• Contribute code, documentation, translations, artwork, samples, etc.
• Report defects (bugs) and suggestions for enhancement.
• Participate in the process of reviewing contributions by others.

If you want to participate in the project development, check out the how to contribute guideline in advance.

Contributors who show dedication and skill are rewarded with additional rights and responsibilities. Their opinions weigh more when decisions are made, in a fully meritocratic fashion.

Maintainer

A Maintainer is a Contributor who is also responsible for knowing, directing and anticipating the needs of a given Module. As such, Maintainers have the right to set the overall organization of the source code in the Module, and the right to participate in the decision-making. Maintainers are required to review the contributor’s requests and decide whether to accept or not.

Name	E-Mail
Jaeku Yun	[e-mail]
Shinhyung Choi	[e-mail]
Roman Babenko	[e-mail]
Yuliia Tatarinova	[e-mail]

How to Contact

Please post questions, issues, or suggestions in issues. This is the best way to communicate with the developers.