Skip to main content

Simple Python library to use the SCANOSS API.

Project description

SCANOSS Scanner

The SCANOSS Scanner is a simple Python script performs a scan of a folder or a WFP file using SCANOSS API.

Usage

Run scanner.py as a Python script, passing as argument the path to the folder to be scanned. You can also install the distribution and use it as a Python module.

Example:

python3 scanoss/scanner.py /path/to/dir/to/scan

scanner.py generates a WFP file that is saved as scan_wfp in the current folder. This file is uploaded to the SCANOSS API, to perform a scan and return the output as in json format.

The complete usage can be seen by using the -h flag.

% scanner.py                                              
usage: scanner.py [-h] [--url URL] [--wfp WFP] [--identify IDENTIFY] [--blacklist BLACKLIST] [--output OUTPUT] [--format {plain,spdx,spdx_xml,cyclonedx}] [--obfuscate] [--summary] [--key KEY] [--apiurl APIURL] [DIR]

Simple scanning agains SCANOSS API.

positional arguments:
  DIR                   A folder to scan

optional arguments:
  -h, --help            show this help message and exit
  --url URL             Scan a URL. It supports urls containing zip files of projects, and it can download master.zip of open projects from GitHub and Gitee
  --wfp WFP             Scan a WFP File
  --identify IDENTIFY   Scan and identify components in SBOM file
  --blacklist BLACKLIST
                        Scan and blacklist components in SBOM file
  --output OUTPUT, -o OUTPUT
                        Optional name for the result file.
  --format {plain,spdx,spdx_xml,cyclonedx}, -f {plain,spdx,spdx_xml,cyclonedx}
                        Optional format of the scan result
  --obfuscate, -p       Obfuscate file names. WARNING: Obfuscation affects the scan results accuracy.
  --summary, -s         Generate a component summary of the scan
  --key KEY, -k KEY     SCANOSS API Key token
  --apiurl APIURL       SCANOSS API URL (overrides default value: https://osskb.org/api/scan/direct)

Installation via pip

You can also install scanner.py via pip: pip3 install scanoss-scanner

Scanning URL

By Default, scanner.py uses the API URL endpoint for SCANOSS OSS KB: https://osskb.or/api/scan/direct. You can change this by setting the environment variable SCANOSS_SCAN_URL to the appropriate SCANOSS API Endpoint. You can also configure the SCANOSS API token using the environment variable SCANOSS_API_KEY.

Winnowing

SCANOSS implements an adaptation of the original winnowing algorithm by S. Schleimer, D. S. Wilkerson and A. Aiken as described in their seminal article which can be found here: https://theory.stanford.edu/~aiken/publications/papers/sigmod03.pdf

The winnowing algorithm is configured using two parameters, the gram size and the window size. For SCANOSS the values need to be:

  • GRAM: 30
  • WINDOW: 64

The result of performing the Winnowing algorithm is a string called WFP (Winnowing FingerPrint). A WFP contains optionally the name of the source component and the results of the Winnowing algorithm for each file.

EXAMPLE output: test-component.wfp

component=f9fc398cec3f9dd52aa76ce5b13e5f75,test-component.zip
file=cae3ae667a54d731ca934e2867b32aaa,948,test/test-file1.c
4=579be9fb
5=9d9eefda,58533be6,6bb11697
6=80188a22,f9bb9220
10=750988e0,b6785a0d
12=600c7ec9
13=595544cc
18=e3cb3b0f
19=e8f7133d
file=cae3ae667a54d731ca934e2867b32aaa,1843,test/test-file2.c
2=58fb3eed
3=f5f7f458
4=aba6add1
8=53762a72,0d274008,6be2454a
10=239c7dfa
12=0b2188c9
15=bd9c4b10,d5c8f9fb
16=eb7309dd,63aebec5
19=316e10eb
[...]

Here, component is the MD5 hash and path of the component (It could be a path to a compressed file or a URL). file is the MD5 hash, file length and file path being fingerprinted, followed by a list of WFP fingerprints with their corresponding line numbers.

Requirements

Python 3.5 or higher.

The dependencies can be found in the requirements.txt file. To install dependencies:

pip3 install -r requirements.txt

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

scanoss-scanner-1.7.3.tar.gz (10.5 kB view details)

Uploaded Source

Built Distribution

scanoss_scanner-1.7.3-py3-none-any.whl (24.1 kB view details)

Uploaded Python 3

File details

Details for the file scanoss-scanner-1.7.3.tar.gz.

File metadata

  • Download URL: scanoss-scanner-1.7.3.tar.gz
  • Upload date:
  • Size: 10.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.3.0 pkginfo/1.6.1 requests/2.25.1 setuptools/54.2.0 requests-toolbelt/0.9.1 tqdm/4.54.1 CPython/3.9.1

File hashes

Hashes for scanoss-scanner-1.7.3.tar.gz
Algorithm Hash digest
SHA256 c0dae65c7f405894e67811592fc4d0be3d6cc7a690b643fba6ae5393fc26c0e7
MD5 df3c3c552da7b588d38d31b5141cd6bb
BLAKE2b-256 2a15853c18116cbb686e8d368df9e7aaaecde77c970a9c60fb951e7b96262b16

See more details on using hashes here.

File details

Details for the file scanoss_scanner-1.7.3-py3-none-any.whl.

File metadata

  • Download URL: scanoss_scanner-1.7.3-py3-none-any.whl
  • Upload date:
  • Size: 24.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.3.0 pkginfo/1.6.1 requests/2.25.1 setuptools/54.2.0 requests-toolbelt/0.9.1 tqdm/4.54.1 CPython/3.9.1

File hashes

Hashes for scanoss_scanner-1.7.3-py3-none-any.whl
Algorithm Hash digest
SHA256 f8f0310d314fbaf0504341a29b09478166f70d297d0baa564e3580e5e02e9313
MD5 55f75493338df6da02cac318ccc0ec12
BLAKE2b-256 c82057b34fe1d6ee787ccb1577293a69d25505234042b04947a43f37b1ce199b

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page