Mend SCA Cleanup Tool
Project description
Mend Projects Cleanup CLI Tool
- The self-hosted CLI tool features cleaning up projects and generating reports before deletion in 2 modes:
- By stating OperationMode=FilterProjectsByUpdateTime and how many days to keep (-r/ DaysToKeep=)
- By stating OperationMode=FilterProjectsByLastCreatedCopies and how many copies to keep (-r/ DaysToKeep=)
- The reports are saved in the designated location as follows: [Output_DIR]/[PRODUCT NAME]/[PROJECT NAME]/[REPORT NAME]
- The default location is the [WORKING DIRECTORY]/Mend/Reports/[PRODUCT NAME]/[PROJECT NAME]/[REPORT NAME]
- To review the outcome before actual deletion use -y true / DryRun=True flag. It will NOT delete any project nor create reports
- By default, the tool generates all possible project-level reports. By specifying ((-t / Reports=/) it is possible to select specific reports
- The full list of available reports is below
- The full parameters list is available below
- There are two ways to configure the tool:
- By configuring params.config on the executed dir or passing a path to the file in the same format
- By setting command line parameters as specified in the usage below
Supported Operating Systems
- Linux (Bash): CentOS, Debian, Ubuntu, RedHat
- Windows (PowerShell): 10, 2012, 2016
Pre-requisites
- Python 3.8+
Permissions
- The user used to execute the tool has to have "Organization Administrator" or "Product Administrator" on all the maintained products and "Organization Auditor" permissions.
- It is recommended to use a service user.
Installation and Execution from PyPi (recommended):
- Install by executing:
pip install mend-sca-cleanup-tool
- Configure the appropriate parameters either by using the command line or in
params.config
. - Execute the tool (
mend_sca_cleanup_tool ...
).
Installation and Execution from GitHub:
- Download and unzip mend-sca-cleanup-tool.zip from the most recent tagged release.
- Install requirements:
pip install -r requirements.txt
- Configure the appropriate parameters either by using the command line or
params.config
. - Execute:
python sca_cleanup_tool.py <CONFIG_FILE>
Examples:
Perform dry run check-in to get to know which projects would have been deleted:
mend_sca_cleanup_tool -r 30 -m FilterProjectsByUpdateTime -u <USER_KEY> -k <ORG_TOKEN> -y true
Keep the last 60 days on each product, omitting a product token <PRODUCT_1> from analyzing:
mend_sca_cleanup_tool -r 60 -m FilterProjectsByUpdateTime -u <USER_KEY> -k <ORG_TOKEN> -e <PRODUCT_TOKEN_1>
Keep only two of the newest projects in each product token PRODUCT_1 and PRODUCT_2:
mend_sca_cleanup_tool -r 2 -m FilterProjectsByLastCreatedCopies -u <USER_KEY> -k <ORG_TOKEN> -i <PRODUCT_TOKEN_1>,<PRODUCT_TOKEN_2>
Analyze only the projects that have the specified Mend tag and keep the newest project in each product:
mend_sca_cleanup_tool -r 1 -m FilterProjectsByLastCreatedCopies -u <USER_KEY> -k <ORG_TOKEN> -g <KEY>:<VALUE>
Keep the last 2 weeks and analyze only the projects whose match their tag key and the tag value contains the specified value:
mend_sca_cleanup_tool -r 14 -m FilterProjectsByUpdateTime -u <USER_KEY> -k <ORG_TOKEN> -v <KEY>:<VALUE>
Keep the last 100 days for both PRODUCT_1 and PRODUCT_2, but do not delete the project PROJECT_1 (which is a project in one of the included products):
mend_sca_cleanup_tool -r 100 -m FilterProjectsByUpdateTime -u <USER_KEY> -k <ORG_TOKEN> -i <PRODUCT_TOKEN_1>,<PRODUCT_TOKEN_2> -x <PROJECT_TOKEN_1>
Keep the last month for both PRODUCT_1 and PRODUCT_2, but do not delete projects that contain provided strings in their names:
mend_sca_cleanup_tool -r 31 -m FilterProjectsByUpdateTime -u <USER_KEY> -k <ORG_TOKEN> -i <PRODUCT_TOKEN_1>,<PRODUCT_TOKEN_2> -n CI_,-test
Full Usage flags:
usage: ws_cleanup_tool [-h] -u MEND_USER_KEY -k MEND_TOKEN [-a MEND_URL] [-t REPORT_TYPES] [-m {FilterProjectsByUpdateTime,FilterProjectsByLastCreatedCopies}] [-o OUTPUT_DIR] [-e EXCLUDED_PRODUCT_TOKENS] [-i INCLUDED_PRODUCT_TOKENS]
[-g ANALYZED_PROJECT_TAG] [-r DAYS_TO_KEEP] [-p PROJECT_PARALLELISM_LEVEL] [-y DRY_RUN]
Mend Cleanup Tool
optional arguments:
-h, --help show this help message and exit
-u MEND_USER_KEY, --userKey
Mend User Key
-k MEND_API_TOKEN, --apiToken, --orgToken
Mend Organization Key (API Key)
-a MEND_URL, --mendUrl, --wsURL
Mend URL. This value defaults to saas.whitesourcesoftware.com.
-t REPORT_TYPES, --reportTypes
Report Types to generate (comma seperated list)
-m OPERATION_MODE, --operationMode {FilterProjectsByUpdateTime,FilterProjectsByLastCreatedCopies}
Cleanup operation mode
-o OUTPUT_DIR, --outputDir
Output directory
-e EXCLUDED_PRODUCT_TOKENS, --excludedProductTokens
List of excluded products
-i INCLUDED_PRODUCT_TOKENS, --includedProductTokens
List of included products
-g ANALYZED_PROJECT_TAG, --AnalyzedProjectTag
Analyze only the projects whose contain the specific Mend tag (key:value)
-v ANALYZED_PROJECT_TAG_REGEX_IN_VALUE, --AnalyzedProjectTagRegexInValue
Analyze only the projects whose match their tag key and the tag value contains the specified value (key:value)
Note: This was originally broken in the original ws-cleanup-tool. The functionality was adjusted to work as originally written. The naming convention is a misnomer but was kept to avoid breaking existing integrations.
-r DAYS_TO_KEEP, --DaysToKeep
Number of days to keep in FilterProjectsByUpdateTime or number of copies in FilterProjectsByLastCreatedCopies
-p PROJECT_PARALLELISM_LEVEL, --ProjectParallelismLevel
Project parallelism level
Note: This is currently not used in this version of the mend-sca-cleanup-tool. Was kept to prevent breaking existing integrations.
-y DRY_RUN, --DryRun
Logging the projects that are supposed to be deleted without deleting and creating reports
default False
-s SKIP_REPORT_GENERATION, --SkipReportGeneration
Skip report generation step
default False
-j SKIP_PROJECT_DELETION, --SkipProjectDeletion
Skip project deletion step
default False
-x EXCLUDED_PROJECT_TOKENS, --excludedProjectTokens
List of excluded projects
-n EXCLUDED_PROJECT_NAME_PATTERNS, --excludedProjectNamePatterns
List of excluded project name patterns
Available reports
The following Mend project reports are available through the clean-up tool. These values can be specified with the -t flag to generate specific reports.
- alerts
- alerts_rejected_by_policy
- attribution
- bugs
- due_diligence
- ignored_alerts
- in_house_libraries
- inventory
- license_compatibility
- resolved_alerts
- request_history
- source_files
- source_file_inventory
- vulnerability
SAST Clean up
If you need to run a clean up script for your SAST environment, please refer to the Mend SAST clean up kit in the Mend Toolkit
note: The optimal cleanup scope is derived from the size of the environment, Mend scope size (memory and CPU) allocated for the server, and runtime time constraints.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
Hashes for mend_sca_cleanup_tool-23.6.1.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3da1486b43e905a6c597f7779657808cf947e3af68c86f263df393e43c729d8e |
|
MD5 | 1fc0dbf725491c3b92994b379f45e90a |
|
BLAKE2b-256 | e9e432f59967e59cd359050f6bae62d8496592ffbd56930c34c1cd1046297c32 |