Community Github Action to scan for SCA using Synopsys Black Duck.
Project description
Community Black Duck GitHub Scan Action
License & Warranty
This is a community supported GitHub Action for launching Black Duck SCA (OSS vulnerability analysis) scans as part of a GitHub CI/CD action workflow. It is provided under an OSS license (specified in the LICENSE file) without warranty or liability and has been developed by Synopsys field engineers as a contribution to the Synopsys user community.
Please direct questions and comments to the Black Duck Integrations Forum in the Synopsys user community.
Overview
The Black Duck direct scan utility supports multiple use cases:
- Run Black Duck Synopsys Detect Full (intelligent) or Rapid scans within GitHub Actions
- For
Pull Requests
, leave comments on a pull request using Rapid scan (dependencies only) to identify direct dependencies violating security policies and offer upgrade guidance including for vulnerable child dependencies (all supported package managers) - For
Commits/Pushes
, create new fix pull requests using Rapid scan (dependencies only) to identify available upgrades for direct dependencies violating security policies including for vulnerable child dependencies (primary package managers only) - Optionally report only newly introduced security policy violations (as compared against the last FULL intelligent scan)
- Optionally export Black Duck security policy violations via SARIF using Rapid scan (dependencies only); for subsequent import as code scanning alerts in GitHub (requires Advanced Security subscription in GitHub)
- Fail the check if security policies are violated
Black Duck RAPID scan policies are used to determine direct dependencies which violate security policies, allowing specific vulnerability severities and types to be covered. See the Black Duck User Guide within your server for more information on configuring security policies.
Quick Start Guide
Follow these outline instructions to implement this utility in a GitHub repository as a GitHub Action.
-
Add GitHub repository secrets (
Settings-->Secrets-->Actions
):BLACKDUCK_URL
with the Black Duck server URLBLACKDUCK_API_TOKEN
with the Black Duck API token
-
Create at least 1 security policy for RAPID scan type in the Black Duck server:
- Browse to
Manage-->Policy Management
- Create a new policy
- Ensure
Scan Mode
is set toRapid
(if you wish to use incremental scanning ensure bothRapid
andFull
scan types are specified) - Add
Component Conditions
to check for vulnerabilities (for exampleHighest Vulnerability Score >= 7.0
)
- Browse to
-
Create a new Action in your GitHub repository:
- Select
Actions
tab - Select
set up a workflow yourself
if no Actions already defined, or selectNew Workflow
to add a new Action - Replace the
jobs:
section in the template YAML added by default with the relevantjobs:
section for this Action - see next step
- Select
-
Check the package managers used in your repository:
- If one or more primary package managers are used (Npm, Lerna, Yarn, Pnpm, Nuget, Maven) then use either the docker container or python deployment modes (for primary package managers) - see below
- If one or more secondary package managers are used (including Conan, Conda, Dart, GoLang, Hex, Pypi) then use ONLY the python deployment modes (for primary and secondary package managers) - see below
-
OPTIONAL If you want to define specific project and version names use the action parameters
project: PROJECT
andversion: VERSION
. Note that Rapid scans do not create the Black Duck project version but can compare against previously scanned projects. -
OPTIONAL Add the action parameter
incremental_results: true
(or add the command line option--incremental_results true
in pythin install mode) to only report violations added since the last full run for the Black Duck project with the same name and version (requires policies configured for bothfull
andrapid
modes). The action will report all security policy violations by default. -
Commit the action configuration YAML file (note that the Black Duck Action should run immediately due to the commit of a new file, but there will be no security scan as no package manager file was changed)
-
OPTIONAL Manually run an intelligent (FULL) scan by selecting
Actions-->Select your new workflow-->Click on Run workflow option
within GitHub -
Thereafter, where a package manager config file is changed within a Pull Request or Commit/push on the master/main branch, the Black Duck Action should scan for security policy violations and update comments or create Fix PRs
Supported Technologies
The utility supports these primary package managers:
- Npm/Lerna/Pnpm/Yarn
- Maven
- NuGet
Repositories built with one or more of the primary package managers can utilise all features, run as a GitHub Action either using the pre-built container or python installation deployment modes. Upgrade guidance will be calculated using Rapid scan (dependencies only) for all security policy violations within both direct and indirect (transitive) dependencies. The action can create comments within Pull Requests or create fix PRs within commits/pushes to address security policy violations.
For projects built with at least one of the following secondary package managers, the action must be installed as a python/PyPi module and run directly as a command (see below). Direct and transitive security policy violations will be reported, but upgrade guidance for all children will not be calculated, and the action will not support creating fix PRs.
- Conan
- Conda
- Dart
- GoLang
- Hex
- Pypi
The utility can support multiple package managers in a single project, although you need to ensure you choose the correct deployment mode (for primary or secondary package managers) based on the full list. For example, if you have a project using Maven
, npm
and Pypi
, you will need to use the python (secondary package manager) deployment mode throughout.
The following table shows the functionality available for the supported package managers:
Package Manager | Comment on Pull Request | Create Fix PRs for vulnerable direct dependencies | Output SARIF for code security check | Run intelligent (full) scan | Action Deployment modes |
---|---|---|---|---|---|
Event Type: pull_request | Event Type: push | Event Types: all | Event Types: all | ||
Scan Type: rapid | Scan Type: rapid | Scan Type: rapid | Scan Type: intelligent | ||
npm | yes | yes | yes | yes | Docker or Python install |
lerna | yes | yes | yes | yes | Docker or Python install |
yarn | yes | yes | yes | yes | Docker or Python install |
pnpm | yes | yes | yes | yes | Docker or Python install |
nuget | yes | yes | yes | yes | Docker or Python install |
maven | yes | yes | yes | yes | Docker or Python install |
conan | yes | yes | yes | Python install | |
conda | yes | yes | yes | Python install | |
dart | yes | yes | yes | Python install | |
golang | yes | yes | yes | Python install | |
hex | yes | yes | yes | Python install | |
pypi | yes | yes | yes | Python install |
Configuration
Prerequisites
For all deployment modes, the following prerequisites are required:
- This utility requires access to a Black Duck Professional server v2021.10 or above.
- At least 1 security policy for RAPID scan must be configured (otherwise scans will show zero results as no components will violate policies).
- The following repository secrets must be configured:
- BLACKDUCK_URL - full URL to Black Duck server (e.g.
https://server.blackduck.synopsys.com
) - BLACKDUCK_API_TOKEN - Black Duck API Token including scan permissions
- BLACKDUCK_URL - full URL to Black Duck server (e.g.
- Ensure additional options to run successful Synopsys Detect dependency scans have been specified (either as environment variables or using the
detect_opts
parameter). For example, you may need to modify the package manager search depth, or exclude specific package managers.
For the python deployment mode (for both primary and secondary package managers):
- Only Linux runners are supported
- Ensure the required package manager(s) are installed and available on the PATH within the Action
Modes of Operation
The action supports several activities:
- On a manual workflow (GitHub event
workflow_dispatch
), run a full (intelligent) scan - For a Pull Request (GitHub
pull_request
event), if there are security policy violations, use Rapid scan (dependencies only) and add a comment with information on the policy violations and set the check status (all supported package managers) - For a Commit/Push (GitHub
push
event), if there are security policy violations, use Rapid scan (dependencies only) to create fix Pull Requests to upgrade the vulnerable direct dependencies (only for the primary package managers listed above) and set the check status - For any activity, if there are security policy violations, use Rapid scan (dependencies only) to create a SARIF output file for import as code security issues in Github (all supported package managers)
Example complete YAML samples have been provided at the end of this document to demonstrate all modes combined.
Deployment Modes
The action can either run as a Docker container which is downloaded dynamically (for primary package managers) or as a python package installed locally (for both primary and secondary package managers), and supports GitHub-hosted and Self-hosted Linux runners.
Run FULL (intelligent) Scan as manual workflow - Docker deployment mode (only for Primary Package Managers)
This step will allow you to run a full scan manually.
Black Duck Full (intelligent) scans support all scan types and create a project/version in the Black Duck server.
Full scans can be used as a baseline to compare subsequent Rapid scans (use the incremental_results
parameter within this utility to show differences since the last Full scan).
Use the YAML step below to support manual Full scans:
- name: Run Baseline Black Duck Scan (manual, workflow dispatch)
if: ${{github.event_name == 'workflow_dispatch'}}
uses: synopsys-sig-community/blackduck-direct-scan-action@v1
with:
bd_url: ${{ secrets.BLACKDUCK_URL }}
bd_token: ${{ secrets.BLACKDUCK_API_TOKEN }}
mode: intelligent
env:
GITHUB_TOKEN: ${{ github.token }}
Add the following Action step configuration:
Check Pull Request or Push - Docker Deployment Mode (only for Primary Package Managers)
The utility will support creating comments on Pull Requests or creating fix PRs to address security policy violations for all dependencies for the primary package managers. The action will also fail the code scan check.
A Black Duck Rapid scan will be run which only imports dependencies and does not create/modify a Black Duck project.
Use the following Action step configuration:
- name: Black Duck security scan
uses: synopsys-sig-community/blackduck-direct-scan-action@v1
with:
url: ${{ secrets.BLACKDUCK_URL }}
token: ${{ secrets.BLACKDUCK_API_TOKEN }}
upgrade_major: true
env:
GITHUB_TOKEN: ${{ github.token }}
You may also need to add the action parameter bd_trustcert: true
to trust the server SSL certificate if not authenticated. See below for full descriptions of all available parameters.
Creating SARIF for Import as GitHub Code Scanning Alerts - Docker deployment mode (for Primary Package Managers)
The utility will create a GitHub SARIF output file of security policy violations for all dependencies for the primary package managers listed above.
A Black Duck Rapid scan will be run which only imports dependencies and does not create/modify a Black Duck project.
The sarif
parameter is used to indicate that a SARIF file should be created. Note that specifying the sarif
parameter will stop the other operation modes (fix_pr
or comment_on_pr
) from running automatically. See the FAQs below for how to run the other operation modes in addition to SARIF output.
Use the YAML step below to create the SARIF file blackduck-sarif.json
:
- name: Black Duck security scan SARIF
uses: synopsys-sig-community/blackduck-direct-scan-action@v1
with:
bd_url: ${{ secrets.BLACKDUCK_URL }}
bd_token: ${{ secrets.BLACKDUCK_API_TOKEN }}
upgrade_major: true
sarif: blackduck-sarif.json
env:
GITHUB_TOKEN: ${{ github.token }}
To import the SARIF file as code scanning alerts you would need an additional YAML step:
- name: "Check file existence"
id: check_files
uses: andstor/file-existence-action@v1
with:
files: "blackduck-sarif.json"
- name: Upload SARIF file
if: steps.check_files.outputs.files_exists == 'true'
uses: github/codeql-action/upload-sarif@v1
with:
sarif_file: blackduck-sarif.json
Python deployment mode (for both Primary and Secondary Package Managers)
If you are scanning a project which uses at least one secondary package manager (see list above), then you need to deploy this utility as a Python package. The fix Pull Request operation mode is not supported for secondary package managers, and any upgrade guidance is limited to the individual package (will not include upgrading any vulnerable child dependencies).
A Black Duck Rapid scan will be run which only imports dependencies and does not create/modify a Black Duck project.
The following YAML extract will add the scan utility as a step running as a python package installed locally:
- name: Set up Python 3.9
uses: actions/setup-python@v2
with:
python-version: 3.9
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -i https://test.pypi.org/simple/ blackduck_direct_scan_action
- name: Run DirectGuidance Scan
run: |
blackduck-direct-scan-action --bd_url ${{ secrets.BLACKDUCK_URL }} --bd_token ${{ secrets.BLACKDUCK_API_TOKEN }} --upgrade_major true
env:
# Pass the GitHub token to the script in order to create PRs
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
You may also need to add the option --bd_trustcert true
to the blackduck-direct-scan-action
command to trust the server SSL certificate if not signed. See below for full descriptions of all available parameters.
All Supported Parameters
The utility action provides a number of input parameters that can be passed using with
or added as options to the blackduck-direct-scan-action
command. Some input parameters have default values that should ensure default behavior if not specified.
Property | Default | Description |
---|---|---|
bd_url | REQUIRED The Black Duck server URL (for example https://server.blackduck.synopsys.com ) |
|
bd_token | REQUIRED The Black Duck API token (create under User-->My Access Tokens in the server UI) |
|
bd_trustcert | false | Trust the certificate from the BD server (required if certificate is not fully signed) |
mode | rapid | Run either an intelligent scan (comprehensive, will update project version in BD server) or rapid scan (runs in seconds, ephemeral - use this to support the main functions of this action) |
project | Black Duck project name. Not required for Rapid scans, but can be specified for BOM comparison against a previous full scan when incremental_results is set to true. |
|
version | Black Duck version name. Not required for Rapid scans, but can be specified for BOM comparison against a previous full scan when incremental_results is set to true. |
|
comment_on_pr | false | Leave a comment on the pull request with the reported issues - if specified and set to true, will override the automatic detection of the event type and stop fix PRs from being created |
fix_pr | false | Generate a fix pull request if a vulnerable component has an available upgrade path; if specified and set to true, will override the automatic detection of the event type and stop PR comments from being created |
upgrade_major | false | Include upgrades that are beyond the current major version of the component being used - note, this can introduce a breaking change if the component's APIs are sufficiently different |
sarif | blackduck-sarif.json | Output results in SARIF file suitable for import into GitHub as code scanning alerts |
incremental_results | false | Set to true to filter the output to only report on newly introduced components (uses the --detect.blackduck.rapid.compare.mode=BOM_COMPARE option and compares configured policies against the previous full scan) |
output_folder | blackduck-output | Temporary location to create output scan data (will be deleted after scan completion |
debug | 0 | Set to value 9 to see debug messages from the action |
no_files_check | false | Skip the validation of the changed files - by default this check will terminate the action if no package manager config files have been changed in the commit/pull request |
detect_opts | Specify Synopsys Detect scan options in a comma-delimited list without leading hyphens (e.g. detect.detector.buildless=true,detect.maven.buildless.legacy.mode=false ) |
Getting Support
For questions and comments, please contact us via the Black Duck Integrations Forum. Issues can also be raised in GitHub.
Specify the action parameter debug: 9
to output full logs from the action run and include logs within the issue or community post.
Overall Example Yaml: Docker Deployment mode (for Primary Package Managers)
The following YAML file shows the configuration of the scan action for primary package managers including the ability to run a full (intelligent) scan manually:
name: Scan a project with Black Duck
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
workflow_dispatch:
jobs:
blackduck:
runs-on: ubuntu-latest
steps:
- name: Checkout the code
uses: actions/checkout@v2
# Runs a Black Duck intelligent scan manually
# This will run a "full" or "intelligent" scan, logging new components in the Black Duck Hub server
# in order to provide real time notifications when new vulnerabilities are reported.
- name: Run Baseline Black Duck Scan (manual, workflow dispatch)
if: ${{github.event_name == 'workflow_dispatch'}}
uses: synopsys-sig-community/blackduck-direct-scan-action@v1
with:
bd_url: ${{ secrets.BLACKDUCK_URL }}
bd_token: ${{ secrets.BLACKDUCK_API_TOKEN }}
mode: intelligent
env:
GITHUB_TOKEN: ${{ github.token }}
# Runs a Black Duck rapid scan for pull request/commit/push
- name: Run Black Duck security scan on PR/commit/push
if: ${{github.event_name != 'workflow_dispatch'}}
uses: synopsys-sig-community/blackduck-direct-scan-action@v1
with:
bd_url: ${{ secrets.BLACKDUCK_URL }}
bd_token: ${{ secrets.BLACKDUCK_API_TOKEN }}
upgrade_major: true
env:
# Pass the GitHub token to the script in order to create PRs
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Overall Example Yaml: Python Deployment Mode (for Primary and Secondary Package Managers)
The following YAML file shows the usage of the scan action for secondary package managers including the ability to run a full (intelligent) scan manually:
name: Scan a project with Black Duck
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
workflow_dispatch:
jobs:
blackduck:
runs-on: ubuntu-latest
steps:
- name: Checkout the code
uses: actions/checkout@v2
# Install Python 3.9 for Black Duck Action
- name: Set up Python 3.9
uses: actions/setup-python@v2
with:
python-version: 3.9
# Install Dependencies for Black Duck Action
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -i https://test.pypi.org/simple/ blackduck_direct_scan_action
# Run manual full/intelligent scan
- name: Run Black Duck Full Scan
if: ${{github.event_name == 'workflow_dispatch'}}
run: |
blackduck-direct-scan-action --bd_url ${{ secrets.BLACKDUCK_URL }} --bd_token ${{ secrets.BLACKDUCK_API_TOKEN }} --mode intelligent
env:
# Pass the GitHub token to the script in order to create PRs
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Run Black Duck rapid scan for pull request/commit/push
- name: Run Black Duck Directguidance Scan
if: ${{github.event_name != 'workflow_dispatch'}}
run: |
blackduck-direct-scan-action --bd_url ${{ secrets.BLACKDUCK_URL }} --bd_token ${{ secrets.BLACKDUCK_API_TOKEN }} --upgrade_major true
env:
# Pass the GitHub token to the script in order to create PRs
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
FAQs
No scan results
The utility supports Rapid (dependency) scan for checking security policy violations and either commenting on Pull Requests, creating Fix PRs on push or generating SARIF outputs.
The Black Duck Rapid dependency scan looks for supported package manager config files in the top-level folder of the repo.
Check the following potential causes:
- The utility wil do nothing unless package manager config files (
pom.xml
,package.json
etc.) have been modified. - Ensure you have created security policy violations configured for RAPID scan mode in the Black Duck server.
- If your project only has config files in sub-folders, use the action parameter
detect_opts: detect.detector.search.depth=1
. Change the depth depending on the folder depth to traverse (for example a value of 1 would indicate depth 1 of sub-folders). - If using the Python deployment mode, check the required package managers are installed in the runner.
- Examine the output and the Synopsys Detect log to see why scanning failed.
- Add the parameter
debug: 9
(docker deployment mode) or--debug 9
command line option (python deployment mode) and rerun
No Black Duck project created by the scan
The utility uses Rapid (dependency) scan for checking security policy violations and either commenting on Pull Requests, creating Fix PRs on push or generating SARIF outputs.
Configure the full (intelligent) scan to create projects in Black Duck. Optionally add scan options to fail on policy violations if you wish to report issues in the pipeline in the Full scan.
No Signature scan run
The utility uses Rapid (dependency) scan for checking security policy violations and either commenting on Pull Requests, creating Fix PRs on push or generating SARIF outputs.
Configure the full (intelligent) scan to run a Signature scan.
Cannot connect to Black Duck server due to certificate issues
Check the bd_url
parameter. Also try setthing the action parameter trustcert: true
to accept the unsigned server certificate.
How to set the BD project/version names in scans - docker mode
The project and version names are not required for Rapid scans unless you want to compare the scan against a previous Full scan.
If you want to specify project and version use the action parameters project: MYPROJECT
and/or version: MYVERSION
in docker deployment mode.
Alternatively add the options --project MYPROJECT --version MYVERSION
to the blackduck-direct-scan-action
command in python install mode.
How to output SARIF and Fix PR or Comment on PR operation modes together
By default the action event-type defines what operation mode will be run.
Specifying the action parameter sarif
or command line option --sarif
will stop the other operation modes from running.
If you wish to output SARIF in addition to comment on PR in the same step, use the following step logic:
- name: Black Duck Rapid security scan for Pull Request
if: ${{github.event_name == 'pull_request'}}
uses: synopsys-sig-community/blackduck-direct-scan-action@v1
with:
bd_url: ${{ secrets.BLACKDUCK_URL }}
bd_token: ${{ secrets.BLACKDUCK_API_TOKEN }}
comment_on_pr: true
upgrade_major: true
sarif: blackduck-sarif.json
env:
GITHUB_TOKEN: ${{ github.token }}
If you wish to output SARIF in addition to fix PR in the same step, use the following step logic:
- name: Black Duck security scan for Pull Request
if: ${{github.event_name == 'push'}}
uses: synopsys-sig-community/blackduck-direct-scan-action@v1
with:
bd_url: ${{ secrets.BLACKDUCK_URL }}
bd_token: ${{ secrets.BLACKDUCK_API_TOKEN }}
fix_pr: true
upgrade_major: true
sarif: blackduck-sarif.json
env:
GITHUB_TOKEN: ${{ github.token }}
Incremental scan using incremental_results option returns no results
This parameter uses the Synopsys Detect BOM_COMPARE mode to compare a Rapid scan against the results of a previous Intelligent (full) scan.
To use this mode, you need to ensure that security policies are configured for both Rapid and Full scan types. See the Synopsys Detect documentation for more details.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for blackduck-direct-scan-action-1.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 132e74f367551c353386ec075bd4a2e661c6f059234f4059e35bf5b8cd80b358 |
|
MD5 | ad603b9c31ef48b189b2d86f18612845 |
|
BLAKE2b-256 | b9d56b6210bab5e702f28ab8c0624db624b7dd4b756dad07fb516f720680d78c |
Hashes for blackduck_direct_scan_action-1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 782e29f7e671f02144ea1b2e8fbaec725bfd85ca04c702f52f6c5078fa389fc6 |
|
MD5 | 9e7ec4f9c811427ef99705515a56db6e |
|
BLAKE2b-256 | b812684ba5de28cde074a95b1f96bff5b351b5e5aa6396beb33062c5d48d3caf |