ws-cleanup-tool 0.3.4

WS Cleanup Tool

WhiteSource Projects Cleanup Tool

Tool to clean up projects from White Source Application.

• The tool generates reports for each project in WhiteSource Organization in 2 modes:
  • By stating OperationMode=FilterProjectsByUpdateTime and how many days to keep (-r/ DaysToKeep=)
  • By stating OperationMode=FilterProjectsByLastCreatedCopies and how many copies to keep (-r/ DaysToKeep=)
• The reports are been saved in a designated location in the form of: [ReportsDir]/[PRODUCT NAME]/[PROJECT NAME]/[REPORT NAME]
• -y true / DryRun=True flag can be used to review the outcome of a run. It will NOT delete any project nor create reports
• By default, the tool generates all possible project level reports. It is possible to state which reports to generate ((-t / Reports=/).
• Full parameters list is available below
• The tool can be configured in 2 modes:
  • By configuring params.config on the executed dir or passing a path to file in the same format.
  • By setting command line parameters as specified in the usage below.

Supported Operating Systems

• Linux (Bash): CentOS, Debian, Ubuntu, RedHat
• Windows (PowerShell): 10, 2012, 2016

Pre-requisites

• Python 3.7+

Permissions

• The user used to execute the tool has to have "Organization Administrator" or "Product Administrator" on all the maintained products and "Organization Auditor" permissions.
• It is recommended to use a service user.

Installation and Execution from PyPi (recommended):

1. Install by executing: pip install ws-cleanup-tool
2. Configure the appropriate parameters either by using the command line or in params.config.
3. Execute the tool (ws_cleanup_tool ...).

Installation and Execution from GitHub:

1. Download and unzip ws-cleanup-tool.zip
2. Install requirements: pip install -r requirements.txt
3. Configure the appropriate parameters either by using the command line or params.config.
4. Execute: python cleanup_tool.py <CONFIG_FILE>

Full Usage flags:

usage: ws_cleanup_tool [-h] -u WS_USER_KEY -k WS_TOKEN [-a WS_URL] [-t REPORT_TYPES] [-m {FilterProjectsByUpdateTime,FilterProjectsByLastCreatedCopies}] [-o OUTPUT_DIR] [-e EXCLUDED_PRODUCT_TOKENS] [-i INCLUDED_PRODUCT_TOKENS]
                    [-g ANALYZED_PROJECT_TAG] [-r DAYS_TO_KEEP] [-p PROJECT_PARALLELISM_LEVEL] [-y DRY_RUN]

WS Cleanup Tool

optional arguments:
  -h, --help            show this help message and exit
  -u WS_USER_KEY, --userKey 
                    WS User Key
  -k WS_ORG_TOKEN, --orgToken
                    WS Organization Key (API Key)
  -a WS_URL, --wsUrl
                    WS URL
  -t REPORT_TYPES, --reportTypes
                    Report Types to generate (comma seperated list)
  -m OPERATION_MODE, --operationMode {FilterProjectsByUpdateTime,FilterProjectsByLastCreatedCopies}
                    Cleanup operation mode
  -o OUTPUT_DIR, --outputDir
                    Output directory
  -e EXCLUDED_PRODUCT_TOKENS, --excludedProductTokens
                    List of excluded products
  -i INCLUDED_PRODUCT_TOKENS, --includedProductTokens
                    List of included products
  -g ANALYZED_PROJECT_TAG, --AnalyzedProjectTag
                    Allows analyze to cleanup only if a project contains the specific WhiteSource tag
  -r DAYS_TO_KEEP, --DaysToKeep
                    Number of days to keep in FilterProjectsByUpdateTime or number of copies in FilterProjectsByLastCreatedCopies
  -p PROJECT_PARALLELISM_LEVEL, --ProjectParallelismLevel
                    Project parallelism level
  -y DRY_RUN, --DryRun
                    Logging the projects that should be deleted without deleting and creating reports
                    default False
  -s SKIP_REPORT_GENERATION, --SkipReportGeneration
                    Skip report generation step
                    default True
  -j SKIP_PROJECT_DELETION, --SkipProjectDeletion
                    Skip project deletion step
                    default False                                        
  -x EXCLUDED_PROJECT_TOKENS, --excludedProjectTokens
                    List of excluded projects
  -n EXCLUDED_PROJECT_NAME_PATTERNS, --excludedProjectNamePatterns
                    List of excluded project name patterns                 

Examples:

# Perform dry run check-in for getting know which projects would have been deleted: 
ws_cleanup_tool -r 30 -m FilterProjectsByUpdateTime -u <USER_KEY> -k <ORG_TOKEN> -y true 
# Keep last 60 days on each product, omitting a product token <PRODUCT_1> from analyzing:
ws_cleanup_tool -r 60 -m FilterProjectsByUpdateTime -u <USER_KEY> -k <ORG_TOKEN> -e <PRODUCT_TOKEN_1>
# Keep only 2 of the newest projects in each product token PRODUCT_1 and PRODUCT_2:
ws_cleanup_tool -r 2 -m FilterProjectsByLastCreatedCopies -u <USER_KEY> -k <ORG_TOKEN> -i <PRODUCT_TOKEN_1>,<PRODUCT_TOKEN_2>
# Analyze only the projects that have a WhiteSource tag and keep the newest project in each product token:
ws_cleanup_tool -r 1 -m FilterProjectsByLastCreatedCopies -u <USER_KEY> -k <ORG_TOKEN> -g <PROJECT_TAG>
# Keep last 100 days for both PRODUCT_1 and PRODUCT_2, but do not delete the project PROJECT_1 (which is a project in one of the included products):
ws_cleanup_tool -r 100 -m FilterProjectsByUpdateTime -u <USER_KEY> -k <ORG_TOKEN> -i <PRODUCT_TOKEN_1>,<PRODUCT_TOKEN_2> -x <PROJECT_TOKEN_1>
# Keep last 100 days for both PRODUCT_1 and PRODUCT_2, but do not delete projects that contain provided strings in their names:
ws_cleanup_tool -r 100 -m FilterProjectsByUpdateTime -u <USER_KEY> -k <ORG_TOKEN> -i <PRODUCT_TOKEN_1>,<PRODUCT_TOKEN_2> -n CI_,-test

note: The optimal number is derived from the size of the environment, WhiteSource scope size (memory and CPU) allocated for the server, and runtime time constraints.