Skip to main content

Simple Python library to use the SCANOSS API.

Project description

SCANOSS Scanner

The SCANOSS Scanner is a simple Python script performs a scan of a folder or a WFP file using SCANOSS API.

Usage

Run scanner.py as a Python script, passing as argument the path to the folder to be scanned. You can also install the distribution and use it as a Python module.

Example:

python3 scanoss/scanner.py /path/to/dir/to/scan

scanner.py generates a WFP file that is saved as scan_wfp in the current folder. This file is uploaded to the SCANOSS API, to perform a scan and return the output as in json format.

The complete usage can be seen by using the -h flag.

% scanner.py                                              
usage: scanner.py [-h] [--url URL] [--wfp WFP] [--identify IDENTIFY] [--blacklist BLACKLIST] [--output OUTPUT] [--format {plain,spdx,spdx_xml,cyclonedx}] [--obfuscate] [--summary] [--key KEY] [--apiurl APIURL] [DIR]

Simple scanning agains SCANOSS API.

positional arguments:
  DIR                   A folder to scan

optional arguments:
  -h, --help            show this help message and exit
  --url URL             Scan a URL. It supports urls containing zip files of projects, and it can download master.zip of open projects from GitHub and Gitee
  --wfp WFP             Scan a WFP File
  --identify IDENTIFY   Scan and identify components in SBOM file
  --blacklist BLACKLIST
                        Scan and blacklist components in SBOM file
  --output OUTPUT, -o OUTPUT
                        Optional name for the result file.
  --format {plain,spdx,spdx_xml,cyclonedx}, -f {plain,spdx,spdx_xml,cyclonedx}
                        Optional format of the scan result
  --obfuscate, -p       Obfuscate file names. WARNING: Obfuscation affects the scan results accuracy.
  --summary, -s         Generate a component summary of the scan
  --key KEY, -k KEY     SCANOSS API Key token
  --apiurl APIURL       SCANOSS API URL (overrides default value: https://osskb.org/api/scan/direct)

Installation via pip

You can also install scanner.py via pip: pip3 install scanoss-scanner

Scanning URL

By Default, scanner.py uses the API URL endpoint for SCANOSS OSS KB: https://osskb.or/api/scan/direct. You can change this by setting the environment variable SCANOSS_SCAN_URL to the appropriate SCANOSS API Endpoint. You can also configure the SCANOSS API token using the environment variable SCANOSS_API_KEY.

Winnowing

SCANOSS implements an adaptation of the original winnowing algorithm by S. Schleimer, D. S. Wilkerson and A. Aiken as described in their seminal article which can be found here: https://theory.stanford.edu/~aiken/publications/papers/sigmod03.pdf

The winnowing algorithm is configured using two parameters, the gram size and the window size. For SCANOSS the values need to be:

  • GRAM: 30
  • WINDOW: 64

The result of performing the Winnowing algorithm is a string called WFP (Winnowing FingerPrint). A WFP contains optionally the name of the source component and the results of the Winnowing algorithm for each file.

EXAMPLE output: test-component.wfp

component=f9fc398cec3f9dd52aa76ce5b13e5f75,test-component.zip
file=cae3ae667a54d731ca934e2867b32aaa,948,test/test-file1.c
4=579be9fb
5=9d9eefda,58533be6,6bb11697
6=80188a22,f9bb9220
10=750988e0,b6785a0d
12=600c7ec9
13=595544cc
18=e3cb3b0f
19=e8f7133d
file=cae3ae667a54d731ca934e2867b32aaa,1843,test/test-file2.c
2=58fb3eed
3=f5f7f458
4=aba6add1
8=53762a72,0d274008,6be2454a
10=239c7dfa
12=0b2188c9
15=bd9c4b10,d5c8f9fb
16=eb7309dd,63aebec5
19=316e10eb
[...]

Here, component is the MD5 hash and path of the component (It could be a path to a compressed file or a URL). file is the MD5 hash, file length and file path being fingerprinted, followed by a list of WFP fingerprints with their corresponding line numbers.

Requirements

Python 3.5 or higher.

The dependencies can be found in the requirements.txt file. To install dependencies:

pip3 install -r requirements.txt

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

scanoss-scanner-1.7.3.tar.gz (10.5 kB view hashes)

Uploaded Source

Built Distribution

scanoss_scanner-1.7.3-py3-none-any.whl (24.1 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page