hiddenshadow 0.11.1

License + SBOM + dependency analyzer for OSS source trees.

HiddenShadow

License + SBOM + CVE + dependency analyzer for OSS source trees.

A standalone tool that scans any directory tree of cloned source repositories and produces:

• License detection — finds LICENSE/COPYING/NOTICE files; identifies SPDX licenses; classifies them per a configurable policy (permissive / weak-copyleft / strong-copyleft / network-copyleft / source-available / public-domain).
• SBOM — emits CycloneDX 1.5 JSON per repo from package-manager manifests (npm, PyPI, Cargo, Go, Maven, Gradle, Composer).
• Library rollup — aggregates every direct dependency across all repos; flags top recurring deps for license review.
• Network license lookup — resolves declared licenses for thousands of deps via the deps.dev v3 API (with persistent cache).
• CVE scan — queries OSV.dev for vulnerabilities against every (package, version) pair in the rollup.
• Dual-license detection — recognizes sibling LICENSE+COPYING pairs with different SPDXs and SPDX-License-Identifier expressions inside COPYING.md narratives, demoting findings appropriately.
• METADATA-aware suppression — when a fork has a METADATA.yaml that documents the disposition of a non-permissive file, scan findings are demoted to a "Documented in METADATA" info entry. Enables CI to surface only NEW findings since fork import.
• Cleartext-secret detection — flags hardcoded credentials in source/config files: private-key blocks, AWS / GitHub / Google / Stripe / Slack / Twilio / SendGrid / npm / PyPI tokens, connection strings with embedded passwords, JWTs, and generic password = "..." / api_key: '...' style assignments. Matches are redacted in the findings file (only first/last 3 chars kept) so the artifact itself is safe to share. Disable with --no-secrets.
• Findings + recommendations — every observed issue is recorded with severity (critical/high/medium/low/info), location, detail, and a concrete recommended fix.

Owned and maintained by DarkShadowSec LLC. Renamed in v0.4 and split into a standalone tool.

Install

cd "E:\New Apps\HiddenShadow"
pip install -e .

Or run directly without install:

python -m hiddenshadow --help

Commands

hiddenshadow scan <path>          # license + SBOM + finding pipeline
hiddenshadow lookup-licenses ...  # resolve unknown licenses via deps.dev
hiddenshadow scan-cves ...        # OSV vulnerability scan
hiddenshadow classify <SPDX>      # show policy decision for one SPDX id
hiddenshadow licenses             # full policy table
hiddenshadow findings ...         # print last-scan findings

Common runs

# Scan the DarkShadowSec source mirror
hiddenshadow scan "E:\New Apps\DarkShadowSec Source Apps"

# Scan with strict CI-mode — exit non-zero on any new high/critical
hiddenshadow scan "E:\path\to\third_party" --fail-on high

# Force the raw view (don't suppress METADATA-documented findings)
hiddenshadow scan "E:\path\to\third_party" --ignore-metadata

# Resolve licenses for the rollup (network — uses deps.dev)
hiddenshadow lookup-licenses --rollup .\library-rollup.csv

# CVE scan via OSV.dev (network)
hiddenshadow scan-cves --rollup .\library-rollup.csv

Layout the scanner expects

<source_root>/
├── <category>/
│   ├── <repo-1>/         (with LICENSE / METADATA.yaml / .git etc.)
│   └── <repo-2>/
├── <repo-3>/             (or a repo can sit directly under source_root)
└── ...

A repo is detected by the presence of any of: .git, METADATA.yaml, LICENSE, LICENSE.txt, LICENSE.md, COPYING, BUILD.darkshadow.

Output

After a scan, <output_root> (defaults to <source_root>) contains:

<output_root>/
├── _logs/
│   ├── scan-findings.json     # full findings + summaries
│   ├── scan-summary.csv       # per-repo row
│   ├── library-rollup.csv     # every direct dep across all repos
│   └── cve-findings.json      # if scan-cves was run
├── sboms/
│   ├── <repo>.cdx.json
│   └── ...
└── _cache/                    # network-fetch caches; safe to delete
    ├── depsdev/
    └── osv-dev/

Severity guide

Severity	When
critical	network-copyleft (AGPL) or source-available (BUSL/SSPL) license inside an authoritative location of a permissive-licensed repo
high	strong-copyleft (GPL) license in an authoritative location; source files claim non-permissive license; LICENSE missing entirely
medium	weak-copyleft (MPL/LGPL) license inside an authoritative path; root LICENSE present but unrecognized
low	reserved
info	non-permissive license in a docs/test path (likely fixture/data); deps needing license lookup; mixed-but-compatible SPDX headers; METADATA-documented dispositions

Configuration

The license policy table lives at hiddenshadow/licenses.py. To support a new license, add a regex pattern to LICENSE_PATTERNS and a classification entry to LICENSE_CLASS. Restart any installed process after editing.

Extending

Module	Responsibility
hiddenshadow.licenses	SPDX patterns + classification policy
hiddenshadow.manifests	package-manager manifest parsers
hiddenshadow.dual_license	dual-license pattern recognition
hiddenshadow.metadata_aware	METADATA.yaml-driven finding suppression
hiddenshadow.findings	analysis rules — license, SBOM, dep
hiddenshadow.secrets	cleartext-credential / API-key detection
hiddenshadow.sbom	CycloneDX 1.5 emission
hiddenshadow.network	shared HTTP+cache for network calls
hiddenshadow.registry	deps.dev lookup
hiddenshadow.osv	OSV.dev vulnerability scan
hiddenshadow.scan	orchestration
hiddenshadow.cli	click-based CLI

Tests

cd "E:\New Apps\HiddenShadow"
pip install -e ".[test]"
pytest tests/

Known limitations

• No registry-resolution for transitives. We only parse direct deps declared in package-manager manifests. For full transitive coverage, run a real SBOM tool (Syft) against compiled artifacts and feed hiddenshadow scan-cves with the resulting SBOM.
• Maven dep resolution is shallow. Real Maven resolves <dependencyManagement> and properties; we just enumerate explicit <dependency> blocks.
• SPDX header sampling capped at 200 files per repo for performance.
• Heuristic license detection — custom licenses (MITRE, DRL) are recognized but unusual ones may not be — they show as "Root LICENSE present but unrecognized" findings, prompting manual review.

License

HiddenShadow itself is proprietary to DarkShadowSec LLC — see the LICENSE file at the repo root. The OSS projects it scans retain their own licenses; HiddenShadow does not change them.