scan the provided projects by using snyk, whitesource or scantist, and convert the result into scantist's format
Project description
Karby
Prerequisite
- install
pipenv
according to any tutorial. - enter
Karby
main folder, executepipenv shell
, runpipenv install
to install the dependencies. - There is a filed named
.env.example
, copy one and change it to.env
. This is where we put all the system environment to. Every time you initiate thepipenv shell
, the system will read those environment variables. All the credentials, tokens or password should not be directly placed in code, but here in.env
Running
now you should be able to run the script.
The entrance is sca_tool_scan.py
. the full command example is
python sca_tool_scan.py [snyk|scantist|whitesource] [upload|cmd] <any/git/url> -name <this is optional> -output <this is optional>
you may not be able to run this because you don't have snyk credentials in your environment
for parameter details, you should run
python sca_tool_scan.py -h
Basic Idea
Motivation
Basically, vulnerability detection tools are aim to find all the components in a given project as well as their vulnerabilities. But different tools are so different in using. karby
is a project aims to collect all of these vulnerability detection tools and simplify their usage.
Input and output
The input should be a single project URL. It could be a git hub URL(for upload scan) or local project directory(for cmd scan).
The output should be a CSV file in the format of Scantist
. You could see the example in karby/report_format_example
. There should be 2 files, the format is <tool_name>-<component or issue>-<project name>.csv
.
- component file: there are 10 fields in this report, but actually only 2 of them matters:
Library
andVersion
. Just leave the rest empty - issue file: there are 10 fields in this report, but actually only 3 of them matters:
Library
Library Version
andPublic ID
.Public ID
is the CVE public Id, if it is not provided, you can also left it empty.
Scan types
I defined 2 types of scan: cmd
and upload
. They are also called: ci scan
and airgap scan
.
cmd
scan need the user to provide the local well-build project directory, and trigger the scan locally in command line. Then, you can use any method to collect the result. You can download the report by url or just parse the output from command line. We don't care, but keep it simple and guarantee the final format(Scantist format)upload
scan need the user to provide the github url to trigger the scan. Or say, trigger by calling APIs given by that tool. This method do not require local build, so we call this **un-build ** project scan asairgap scan
. Not all tools support thisupload
scan method. Need an example? You can refer to snyk api scan.
Scan Tools
Whitesource
Mode avaliable
cmd
: supportupload
: support (this will not trigger a new scan, but find the project online by given name)
Prerequisite
- One of the following Java versions:
- Java JDK 8
- Java JRE 8
- Java JDK 11
- Depending on your project type, ensure that the relevant package managers are installed. You can refer to "Prerequisites" section in the following link for more details.
Setup
- Download wss-unified-agent.jar from the following links:
Environment Variables
WHITESOURCE_API_KEY
: WhiteSource API key, a unique identifier of your organization. Should be a 256-bit hex number.WHITESOURCE_USER_KEY
: WhiteSource User Key. Can be generated from the Profile page in WhiteSource account. In order to get a scan report, the user key must be generated by administrators. Should be a 256-bit hex number.WHITESOURCE_PRODUCT_NAME
: Name of your product.PATH_TO_WHITESOURCE_JAR
(cmd
mode only): Path to the Whitesource local agent .jar file (e.g. /path/to/jar/wss-unified-agent.jar)PATH_TO_WHITESOURCE_CFG
(cmd
mode only, optional): Path to the Whitesource local agent .config file (e.g. /path/to/cfg/wss-unified-agent.cfg). We put a prepared one inside folderesources
. If you need any special modification, go toresources/wss-unified-agent.config
snyk
Mode avaliable
cmd
: supportupload
: support
Prerequisite
- install snyk command line tool in the local environment using
npm install -g snyk
- see if snyk is successfully installed
snyk version
Environment Variables
SNYK_USR_TOKEN
: can be retrieved from personal setting: https://app.snyk.io/manage/integrationsSNYK_ORG_ID
: can be retrieved from personal settingSNYK_INTEGRATION_ID
: can be retrieved from personal setting
scantist
Mode avaliable
cmd
: supportupload
: not support
Prerequisite
- download scantist-bom-detect.jar from https://scripts.scantist.com/staging/scantist-bom-detect.jar
Environment Variables
SCANTIST_EMAIL
: scantist account emailSCANTIST_PSW
: scantist account passwordSCANTIST_BASEURL
: target base url, default to "https://api-staging.scantist.io/"SCANTIST_SBD_HOME
: path to scantist-bom-detect.jar
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for scantist_karby-4.1.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 69b528f670a057ff91eff5b8671733a05c8dcb087e1d44c170fbbb9290aad851 |
|
MD5 | dfe625d2d855fef00181a0af8459e450 |
|
BLAKE2b-256 | 6c93565f5f43bca462408e47eee902f3198ee16c9fa30e501eb4accefeec75d1 |