NetScraper

Automates network pentest checks against a list of IPs, hostnames, or CIDR subnets, and produces per-finding POC screenshots sorted into folders — ready to drop straight into a report appendix.

pipx install netscraper
netscraper --targets targets.txt

pipx is the recommended way to install any Python command-line tool (as opposed to a library) — it builds netscraper its own isolated environment instead of installing its dependencies into your system/Homebrew Python, so it can't collide with anything already there, and it puts netscraper on your PATH automatically. Don't have pipx? brew install pipx (macOS) or python3 -m pip install --user pipx (anywhere), then pipx ensurepath once (restart your terminal after). Prefer plain pip instead? pip install netscraper (add --break-system-packages if your Python complains about an "externally managed environment") works too — see Troubleshooting installation below if that route hits a package conflict.

That's it — no separate playwright install chromium step: the first time the tool actually needs Chromium for a POC screenshot, it notices it isn't downloaded yet and fetches it automatically (a one-time, ~150MB download; every run after that is an instant no-op check). For authorized penetration testing and security research only — see the legal banner the tool itself prints on every run. See "Requirements" below for the external command-line tools (sslscan, nmap, etc.) each check shells out to; pip install alone gets you the netscraper command and its Python dependencies, not those OS-level tools.

Runs vuln-first, not host-first: every check's required port (TLS's own port, SSH/22, SMB/445, LDAP/389, RPC/445, MSSQL/1433, FTP/21, HTTP defaulting to 80 but yours to choose -- see "Usage" below) gets port-gated (a fast parallel TCP connect check) across the whole target list in one unified scan, up front, before anything else happens. UDP-only checks (DNS, SNMP) skip the TCP pre-gate entirely — there's nothing meaningful to "nc -z" for a UDP service, so the real protocol probe itself is the gate.

That port scan's real, live open/closed counts then drive an interactive menu — pick TLS, SSH, SMB, ..., any combination, or All — so you run exactly the check(s) that actually apply to this target list instead of always running all ten. Different engagements need different things: a quick TLS-only re-test after a cert renewal doesn't need to sit through SMB/LDAP/RPC checks against hosts that don't even run those services. Once you've picked, only hosts that actually answered on the relevant port get the real protocol-specific probe — a host without SMB open never gets an SMB check attempted, it's recorded N/A, not scanned, not guessed at, and never lumped in with a "safe" result it didn't earn.

When you kick off a scan it immediately tells you how many targets were loaded, then shows the port-scan results, then the mode picker. During a live scan it pauses 5 seconds between checks once one finishes and before the next starts (--mode-pause, default 5, set to 0 to disable) — a courtesy pause between check types, not between individual hosts.

For scripted/automated use, skip the interactive picker entirely with --modes (comma-separated, e.g. --modes tls,ssh, or --modes all).

Scans several hosts at once, automatically

Once a check's port-gating is done, the actual per-host work (running the real tool, saving evidence, rendering the POC) runs against several hosts at the same time instead of one at a time — this is what keeps a target list of dozens or hundreds of hosts from taking as long to scan as it would running host-by-host. There is no flag, prompt, or setting for this anywhere: the moment you give the tool a target list, it works out how many hosts to run in parallel from the machine it's running on (RAM and CPU count) and just does it. You'll see a line like:

[+] Parallel scan workers: 6 (auto-sized for this machine)

right before each check starts, and results stream in as each host finishes — not all at once at the end, and not silently, so a long-running scan against 100+ hosts still shows visible progress the whole way through instead of looking stuck.

This has to be a pool of processes, not threads: the pty-based capture used for sslscan/ssh-audit output (core/terminal_capture.py) calls os.fork() directly, which is unsafe to do from a multi-threaded process, and Playwright's browser automation API is single-thread-only. A process pool sidesteps both, since every worker is a fully independent OS process. Each worker process launches its own Chromium instance for POC rendering the first time it needs one, then reuses that same instance for every remaining screenshot it renders for the rest of the run — the same "launch once, reuse many times" design as a single-process run (see the POCs section below), just scoped per worker process instead of per whole run, since a Playwright browser object can't be handed from one process to another.

The worker count is picked once per run and capped in both directions: at least 2, never more than 8 regardless of how much RAM a machine reports (more than that mostly just means more processes competing for the same network link and the same target host's own connection limits, not real extra throughput), and never more than the number of CPU cores actually available (oversubscribing a 2-core box with 6+ worker processes measurably slowed things down in testing — more workers than cores buys you context- switching overhead, not parallelism). Within that range it's sized from total system RAM (roughly one worker per 1.5 GB, since each worker can be holding open a Chromium instance alongside whatever external tool that check shells out to) — a fairly typical 8–16 GB laptop lands in the 5–8 worker range this way. If total RAM can't be detected at all (unsupported platform), it falls back to a conservative 8 GB assumption rather than guessing high.

A single worker pool is created once per run and reused across every check you pick and every round of the interactive check-menu loop below — so a worker's already-launched Chromium instance keeps paying off as you run TLS, then SSH, then SMB against the same target list, instead of being torn down and relaunched between checks.

The tool never just exits on you

When run interactively (bare netscraper, or with everything except a target given up front), finishing a check drops you straight back into the same check menu, against the same target list — not back to the Individual Mode / Bulk Scanning / Exit menu, and not back to your shell. Run TLS, look at the results, then pick SSH next against the exact same hosts without re-entering a target or a file path a second time. summary.csv and PT_Scan_Report.xlsx accumulate across every round in that same output folder, so by the time you're done picking checks, one report has everything you ran in this session — not one report per check.

When you're actually done with this target list, either type back (or its number, always the entry right after "All checks") at that check menu, or just hit Ctrl-C there — both take you back to the Individual Mode / Bulk Scanning / Exit menu so you can point the tool at something else. The only way the tool actually quits the whole process is choosing Exit at that menu (or Ctrl-C at that exact menu).

Anywhere else — entering a target, picking check modes, or a scan that's actively running — Ctrl-C means "go back", not "crash". Hit it mid-scan and you'll see whatever results were already collected get saved (both summary.csv and PT_Scan_Report.xlsx, same as a normal finish) before you're bounced back to the check menu for this target list. The same save-first behavior also covers the truly unexpected case — a POC render hiccup, or anything else that wasn't already turned into a clean per-host ERROR row — so one bad screenshot on host #47 of a 200-host run costs you that one row, never the rest of the scan or everything found before it.

(This looping behavior is for interactive use only. A scripted invocation with --targets/--target and --modes given on the command line runs that fixed set of checks once and exits, same as always — no menu, no loop — since there's no interactive session to loop back into.)

What it checks

TLS check (SSL_Scan/, gates on each target's own port, default 443):

• Expired SSL Certificate — certificate notAfter date is in the past.
• Deprecated SSL/TLS Protocol(s) Enabled — SSLv2/SSLv3/TLSv1.0/TLSv1.1 accepted.
• Weak Cipher Suite(s) Accepted — sub-128-bit, RC4/DES/3DES/NULL/EXPORT/MD5/anon-*, or legacy SHA-1 MAC suites (*-SHA, not *-SHA256/*-SHA384).
• Weak Certificate Key Strength — RSA/DSA under 2048 bits, EC under 224 bits.
• Certificate Hostname Mismatch — none of the cert's CN/SAN entries match the scanned hostname (skipped when scanning by bare IP).
• Untrusted / Self-Signed SSL Certificate — a genuinely self-signed cert or a complete chain that resolves to a CA the system doesn't trust.
• Incomplete SSL/TLS Certificate Chain — the server fails to send the intermediate certificate(s) needed to build a chain (see "False-positive avoidance" below for how this is told apart from the untrusted-CA case).
• Zero findings → SAFE POC in SSL_Scan/Secure_POC/.

SSH check (SSH/, gates on port 22):

• Weak / Deprecated Cryptographic Configuration (SSH) — anchored on ssh-audit's own "critical" recommendation tier, not its "warning" tier (which also flags purely forward-looking things like post-quantum readiness on an already-patched server).
• SSH Banner / Version Disclosure — near-universal, low-severity finding (any reachable SSH server discloses this pre-auth). A host can't get a pure SAFE verdict from this check for that reason alone.

SMB check (SMB/, gates on port 445):

• SMBv1 Enabled — nmap smb-protocols reports the legacy NT LM 0.12 (SMBv1) dialect accepted.
• SMB Signing Not Enforced — NTLM Relay Exposure — read directly off nmap's own smb-security-mode (legacy/SMB1) or smb2-security-mode (SMB2/3) verdict text: "disabled", or "enabled but not required".
• SMB Null Session / Anonymous Access — a real smbclient -L //host -N null-session share listing actually succeeds (not just an anonymous session being established, which is normal SMB protocol behavior on almost any server — see below).
• Signing status the scripts didn't recognize → Needs Manual Verification, not a guessed Safe/Vulnerable.

LDAP check (LDAP/, gates on port 389):

• LDAP Anonymous Bind Enabled — an unauthenticated ldapsearch bind + base search actually succeeds.
• LDAP Signing Not Enforced — a ldapsearch simple bind with a bogus DN/password over plaintext gets Invalid credentials (49) (the server evaluated the credentials over an unsigned channel) rather than Confidentiality required (13) / Stronger authentication required (8) (the server refused before even looking at credentials — signing enforced). Works with zero valid creds; see "False-positive avoidance".
• Unrecognized bind result → Needs Manual Verification.

RPC check (RPC/, gates on port 445):

• Exposed MSRPC Services / Endpoint Mapper — rpcdump.py (Impacket) enumerates 1+ interfaces registered with the endpoint mapper via an unauthenticated query.
• MS-SCMR and atsvc Exposed over RPC — among whatever's registered, specifically the Service Control Manager Remote Protocol (MS-SCMR) or Task Scheduler/AT-Scheduler (MS-TSCH/atsvc) interfaces.
• A genuine transport failure (as opposed to a clean, well-formed "no endpoints registered" answer) → Needs Manual Verification.
• Honesty note: this check's tooling and parsing are real (validated against a real Samba fileserver's genuine DCERPC responses), but a populated endpoint list (i.e., actually seeing MS-SCMR/MS-TSCH registered) requires a real Windows host or AD DC, which wasn't available to stand up in this build environment. Test against a real Windows target before relying on this for a report.

DNS check (DNS/, UDP, no TCP pre-gate, port 53):

• DNS Recursion Enabled — a dig query for an arbitrary non-local domain comes back with the ra (Recursion Available) flag set.
• No usable response at all → Needs Manual Verification.

MSSQL check (MSSQL/, gates on port 1433):

• MSSQL Service Information Disclosure — nmap ms-sql-info gets back instance name / version / TCP port / clustering info pre-auth.
• Honesty note: no real MS SQL Server was available in this build environment to validate the positive detection path live; the negative path (closed/non-MSSQL port → no disclosure) was validated for real. Verify against a genuine MSSQL instance before relying on this check.

SNMP check (SNMP/, UDP, no TCP pre-gate, port 161):

• Weak / Default SNMP Community String — onesixtyone against a bundled wordlist of common default/weak community strings (wordlists/snmp_communities.txt) gets a real reply for 1+ of them.

FTP check (FTP/, gates on port 21):

• Anonymous FTP Access — a real scripted ftp client session logging in as anonymous gets back 230 Login successful. (not 530 Login incorrect.).

HTTP check (HTTP/, gates on port 80):

• Server / Service Version Disclosure — the Server: response header contains an actual version number (nginx/1.18.0, not just nginx).
• Host Header Injection — request sent with Host: bing.com; flagged only when the response is a redirect (3xx) whose Location header reflects bing.com back (proves the app builds absolute URLs from the attacker-controlled Host header instead of a fixed canonical hostname).
• Cross-Site Tracing (XST) — a TRACE request carrying a unique X-PT-XST-Marker header; flagged only when that exact header is echoed back verbatim in the response body (proves TRACE is enabled and reflects requests — the classic XST primitive for defeating HttpOnly via a co-hosted script).

DIRSEARCH check (Dirsearch/, port(s) you choose, not fixed):

• Unlike every other check, DIRSEARCH has no single fixed port — after you select it (interactively, or via --modes dirsearch), you're asked for one or several ports (e.g. 80 or 80,443,8080; scripted runs pass --dirsearch-ports instead of being prompted), and only those get a quick TCP gate across the target list. Every host:port that answers gets a real dirsearch brute-force run in quiet mode, already asked (via dirsearch's own -i/--exclude-sizes flags) for only status 200/301/302 and never a 0-byte response — then the real captured output is independently re-checked against that exact same filter, so nothing slips through just because a flag didn't do what it was supposed to. No POC screenshot is rendered for this check — the filtered "Found" lines are saved straight to a .txt evidence file (and rolled up into PT_Scan_Report.xlsx the same as every other check: one red "Vulnerable" row per host:port with anything found, one green "Not Vulnerable" row per host:port with nothing).

CORS check (CORS/, port(s) you choose, not fixed):

• Same "no fixed port" shape as DIRSEARCH — pick it (interactively, or via --modes cors), then give it one or several ports (--cors-ports for a scripted run); only those get a quick TCP gate across the target list.
• CORS Misconfiguration — Arbitrary Origin Reflected — every qualifying host:port gets a real curl request carrying Origin: https://evil.com (a domain that could never legitimately be on any real allow-list). Flagged only when the server's own Access-Control-Allow-Origin response header echoes that exact value back verbatim — a static * wildcard, a different real allow-listed origin, or no CORS header at all are all correctly left Safe, since none of those is the server actually reflecting our attacker-controlled input. If Access-Control-Allow-Credentials: true is also present, the finding is raised to high severity (credentialed cross-origin reads — the account-takeover-class variant); without it, medium.
• No POC screenshot is rendered for this check either — the evidence .txt always includes the exact curl command, already filled in with the real host/port, so a tester can re-run it by hand and capture their own screenshot for the report if they want one.

"Needs Manual Verification"

A few checks (SMB signing when nmap can't get a clean read, LDAP signing/anonymous-bind on an unrecognized bind result, RPC endpoint-mapper reachability on a genuine transport failure, DNS recursion on no response) are behaviorally ambiguous by nature — forcing them to a binary Safe/Vulnerable verdict from an inconclusive signal would just manufacture false positives or false negatives. These get their own Needs_Manual_ Verification/<Check_Name>/ subfolder under the check's output directory, with a yellow NEEDS MANUAL VERIFICATION badge on the POC instead of the usual red/green — same real captured evidence, honestly labeled as inconclusive rather than guessed at. summary.csv marks these rows MANUAL_VERIFICATION, distinct from VULNERABLE/SAFE/N/A/ERROR. A host can carry confirmed findings and manual-verification items from the same check at once (e.g. LDAP: anonymous bind confirmed vulnerable, signing flagged for manual verification, both real, independent signals).

False positives are avoided by anchoring on the tool's own real signal, not a guess

• SSH weak crypto anchors on ssh-audit's "critical" bucket specifically — its own maintained security database, not a heuristic reinvented here.
• Untrusted CA vs. incomplete chain are told apart by counting how many certificates the server actually sent (-showcerts): 1 (leaf only) and validation fails = the server's fault (incomplete chain); 2+ (a real chain) and validation still fails = the chain is fine but the root isn't trusted here (untrusted CA).
• SMB anonymous access is judged by whether the null-session share listing actually succeeded, not merely whether an anonymous session was established — establishing one is normal SMB protocol behavior on almost any server (validated against a real hardened local smbd: "Anonymous login successful" still printed even though the subsequent listing was denied with NT_STATUS_ACCESS_DENIED). Judging on session-establishment alone would false-positive on virtually every host.
• LDAP signing is judged by the server's own real LDAP result code from a bind attempt with bogus credentials — Invalid credentials (49) proves the server evaluated the credentials unprotected; Confidentiality required (13) / Stronger authentication required (8) proves it refused before ever looking at them. No valid credentials are needed either way.
• RPC endpoint-mapper reachability distinguishes a genuine, well-formed "no endpoints registered" DCERPC response (ept_s_not_registered — the mapper answered, there's just nothing there) from an actual transport failure (connection refused, timeout) — only the latter goes to manual verification; the former is a real Safe result.
• SNMP / MSSQL absence-of-a-hit is a real Safe result for what those specific checks claim ("weak/default community string" / "info disclosed"), not a blanket "this service is fully secure" claim.
• Host Header Injection / XST are deterministic, binary tests per an exact specified method (Host: bing.com → redirect Location reflects it; TRACE → unique marker header reflected in the body) — no heuristics.
• Every verdict is backed by the tool's own real output — nothing here reformats or reinterprets what the underlying tool said.

Requirements

• Python 3.9+, on macOS or Linux (uses a Unix pty for ANSI-preserving terminal capture where relevant; no Windows support).
• pip install netscraper (once) — pulls in ssh-audit, impacket (for rpcdump.py), dirsearch, playwright (renders POC screenshots), ansi2html (real captured terminal colors → matching HTML), and openpyxl (writes PT_Scan_Report.xlsx) as dependencies automatically, and puts the netscraper command on your PATH. No separate playwright install chromium step needed — see ensure_chromium_installed() in core/poc_renderer.py, which the netscraper command calls on startup and which downloads Chromium the first time it notices it isn't there yet. Running from a source checkout instead of the published package? pip install -e . from the repo root does the same thing against your local copy.
• Command-line tools this toolkit shells out to (install via your package manager — Debian/Ubuntu names shown):
  • sslscan, openssl — TLS check (apt install sslscan; openssl is already on virtually every Mac/Linux box).
  • nmap — SMB and MSSQL checks (apt install nmap).
  • smbclient — SMB check (apt install smbclient).
  • ldapsearch — LDAP check (apt install ldap-utils).
  • rpcdump.py — RPC check, comes from pip install impacket (already in requirements.txt), lands on PATH automatically.
  • dig — DNS check (apt install dnsutils).
  • onesixtyone — SNMP check (apt install onesixtyone); uses the bundled wordlists/snmp_communities.txt. Optional on macOS: this is an apt-only tool with no real Mac package, so if it's not found on PATH the SNMP check automatically falls back to a built-in, dependency-free Python SNMPv1 prober (core/snmp_pure_probe.py) that does the exact same community-string check over a real UDP socket — same wordlist, same logic, no install needed. The POC always discloses honestly which of the two actually ran.
  • ftp — FTP check (apt install ftp).
  • curl — HTTP check (usually already present).
  • dirsearch — DIRSEARCH check; already installed by pip install -r requirements.txt above (or clone https://github.com/maurosoria/ dirsearch and add it to PATH yourself if you'd rather not pip-install it).
• Real network reach to whatever you're scanning. Run this from a machine or jump box that's actually inside/adjacent to the target scope — a sandboxed or heavily-proxied environment (anything that TLS-intercepts outbound traffic) will give you false results for the TLS check specifically, since it'll see its own proxy's certificate instead of the real target's. (Every check in this toolkit was instead validated during development against real, purpose-built local test servers — real sslscan/ssh-audit/nmap/smbclient/ldapsearch/rpcdump.py/ dig/onesixtyone/ftp/curl output, not synthetic data, with one deliberately-vulnerable and one hardened instance per check wherever that distinction matters — see fixtures/ and the per-check "Honesty note" callouts above for the two checks where full live parity with a real Windows/AD or MSSQL target wasn't achievable in that environment.)

Troubleshooting installation

• zsh: command not found: netscraper right after pip install netscraper — almost always means the install itself didn't actually finish (see the next point) rather than a PATH problem; check the output of the install command for an error before assuming PATH is the issue.
• pip install netscraper --break-system-packages fails partway through with something like Cannot uninstall <package> / no RECORD file was found / hint: The package was installed by brew — this happens on macOS when a dependency netscraper needs (commonly typing_extensions or six) was already installed system-wide by Homebrew rather than by pip, and pip refuses to upgrade a package it can't safely account for. This isn't a netscraper packaging problem, and it isn't specific to any one version — it can hit any pip install that happens to need a newer version of whatever Homebrew already put there. Two ways out, in order of preference:
  1. Use pipx install netscraper instead (see above) — it installs into its own isolated environment and never touches your system/Homebrew Python packages at all, so this conflict can't happen.
  2. Or add --ignore-installed to the same pip command you already ran, e.g. pip install netscraper --break-system-packages --ignore-installed — this tells pip to install its own fresh copy of the conflicting package into its own site-packages instead of trying (and failing) to uninstall the Homebrew-owned one first. Either way, run which netscraper afterward to confirm the command actually landed on your PATH before assuming the install worked.

Running the tests

A small, fast, fully offline regression test suite lives in tests/ -- each test guards against a specific real bug found and fixed in this codebase's history (an ANSI-color-code split silently breaking a highlight match, a CORS connection failure being misread as SAFE, a cert-chain evidence wording mistake, ...), so a future edit can't quietly bring one of them back. It never touches the network or shells out to a real tool -- everything runs against synthetic input shaped exactly like the real tool output that triggered each bug.

pip install -e .[dev]   # from a source checkout -- installs pytest too
pytest

POCs are real terminal captures, not summaries

Each POC .png is the actual tool output — sslscan, ssh-audit, nmap, smbclient, ldapsearch, rpcdump.py, dig, onesixtyone, ftp, or curl, whichever check it's from (the exceptions are DIRSEARCH, which by design saves its filtered "Found" results straight to a .txt file, and CORS, which saves the ready-to-run curl PoC command instead — neither ever renders a screenshot) — captured through a pseudo-terminal (core/terminal_capture.py) for the two tools that colorize their output (sslscan, ssh-audit) so coloring isn't suppressed by the lack of a real TTY; every other tool in this toolkit doesn't colorize its output at all, so a plain capture is already the real thing. The captured bytes go through unmodified: real colors map 1:1 to matching HTML colors (osx palette — macOS Terminal.app's default ANSI colors) and nothing in the text itself is summarized, reworded, or filtered. The POC's title bar, prompt line, and footer all show the exact real command that was run — never a hardcoded tool name — so every screenshot honestly reflects what actually produced it.

The only thing added on top is an outline box around the specific line(s) relevant to that finding — red for a confirmed Vulnerable finding, drawn around the real text without ever touching it. A Needs-Manual- Verification POC gets a yellow badge instead of red/green. The Secure_POC card shows the same real output with no box at all.

One Chromium browser instance is launched once and reused for every single POC rendered from that same process — not relaunched per screenshot. A run against a large target list can easily produce several hundred POCs (one per finding, plus one per clean host per check), and launching a fresh browser process for each one doesn't scale: it's slower, and it's a real reliability risk under sustained load. In a single-process run, core/poc_renderer.py's browser_session() launches that one browser for the whole run; when scanning runs in parallel (see "Scans several hosts at once, automatically" above), each worker process instead calls get_process_browser() to lazily launch its own instance the first time it needs one, then keeps reusing that same instance for the rest of the run — a Playwright browser object can't be shared across process boundaries, so the reuse is scoped per worker process rather than per whole run, but it's still one Chromium launch per worker instead of one per screenshot. Either way, the browser is always closed on the way out, exception or not, so a render failure partway through a long scan can never leak the browser process.

Usage

# Scan a list of targets (IPs, hostnames, and/or CIDR subnets, one per line)
# --output defaults to ~/Netscraper/results if omitted -- the same real
# folder every time, regardless of which directory you ran this from (you
# installed this with pip, so there's no "project folder" to default to
# the way running from a source checkout would have). With no --modes
# flag, you'll be prompted to pick which check(s) to run once the port
# scan finishes.
netscraper --targets targets.txt

# Scan a single target or a whole subnet on the command line
netscraper --target 192.168.1.10
netscraper --target 192.168.1.0/28

# Custom port (used by the TLS check; every other check checks its own
# fixed port regardless, e.g. SSH always 22, SMB always 445)
netscraper --target 192.168.1.10:8443

# Skip the interactive picker and run specific check(s) non-interactively
netscraper --targets targets.txt --modes tls,ssh
netscraper --targets targets.txt --modes smb,ldap,rpc
netscraper --targets targets.txt --modes dns,mssql,snmp,ftp,http
netscraper --targets targets.txt --modes all

# DIRSEARCH has no fixed port -- give it one (or several, comma-separated)
# with --dirsearch-ports when running it non-interactively; in the
# interactive picker you're simply asked for it right after choosing DIRSEARCH.
netscraper --targets targets.txt --modes dirsearch --dirsearch-ports 80,443,8080

# CORS is the same shape -- no fixed port, give it one or several with
# --cors-ports (or you'll be asked interactively after picking CORS).
netscraper --targets targets.txt --modes cors --cors-ports 443,8443

# HTTP defaults to port 80 if you don't say otherwise (same as always), but
# now also takes one or several ports with --http-ports, same shape as
# DIRSEARCH/CORS above -- e.g. an app that only listens on 8080.
netscraper --targets targets.txt --modes http
netscraper --targets targets.txt --modes http --http-ports 80,8080,8000

# Skip the 5s pause between checks during a live scan, or change it
netscraper --targets targets.txt --mode-pause 0
netscraper --targets targets.txt --mode-pause 10

At the very start of every run, before anything is scanned, the tool prints how many targets were loaded and exactly which folder this run's evidence is going into, then runs the unified port scan across every check up front, and — unless you passed --modes — asks which check(s) you actually want:

============================================================
[+] Target list loaded: 137 IP/host(es) to scan
[+] Scan type: LIVE
[+] Results folder: /Users/kav1n/Netscraper/results/scan_20260907_143022_a1b2c3
============================================================

[+] Scanning required ports across 137 target(s) ...
    TLS           : 121/137 host(s) open
    SSH           : 40/137 host(s) open
    SMB           : 12/137 host(s) open
    LDAP          : 3/137 host(s) open
    RPC           : 12/137 host(s) open
    DNS    (UDP)  : checked directly during the scan (no TCP pre-check possible)
    MSSQL         : 2/137 host(s) open
    SNMP   (UDP)  : checked directly during the scan (no TCP pre-check possible)
    FTP           : 5/137 host(s) open
    HTTP          : 118/137 host(s) open
    DIRSEARCH     : port(s) chosen after you select this mode
    CORS          : port(s) chosen after you select this mode

Select which check(s) to run:

  [1] TLS       -- 121/137 host(s) with the required port open
  [2] SSH       -- 40/137 host(s) with the required port open
  [3] SMB       -- 12/137 host(s) with the required port open
  [4] LDAP      -- 3/137 host(s) with the required port open
  [5] RPC       -- 12/137 host(s) with the required port open
  [6] DNS       -- UDP service, checked directly
  [7] MSSQL     -- 2/137 host(s) with the required port open
  [8] SNMP      -- UDP service, checked directly
  [9] FTP       -- 5/137 host(s) with the required port open
  [10] HTTP     -- 118/137 host(s) with the required port open
  [11] DIRSEARCH -- directory/file brute-force (pick port(s) after selecting)
  [12] CORS   -- CORS misconfiguration check (pick port(s) after selecting)
  [13] All checks

  Select mode(s) (e.g. 1,3 or 'all'):

Type a number, several comma-separated (1,3,7), or all. Only the check(s) you pick actually run — no more sitting through SMB/LDAP/RPC checks against a target list that's mostly web servers just because the tool insists on doing all ten every time. Picking DIRSEARCH or CORS prompts you for its port(s) right after you make your selection, since those are the two checks with no fixed port of their own.

Every run gets its own results folder

By default, each run writes into a uniquely-named subfolder under --output (default root ~/Netscraper/results — the same real folder on disk every time, no matter which directory you happened to run netscraper from) instead of a single fixed path — e.g. ~/Netscraper/results/scan_20260907_143022_a1b2c3 (a timestamp plus a short random suffix, so two runs started in the same second never collide). This means re-scanning the same targets later (a re-test after remediation, a repeat engagement, etc.) never silently overwrites a previous run's POCs and raw evidence — every run's evidence is kept, side by side, under ~/Netscraper/results/.

If you're scripting this and want full control over the exact output path yourself (e.g. in CI), pass --no-run-id to write directly into --output with no extra subfolder (this WILL overwrite whatever was already at that path):

netscraper --targets targets.txt --output ./results --no-run-id

Replay mode (no live scanning)

For reprocessing previously-captured output, or testing the pipeline without touching the network:

netscraper --targets targets_example_tls.txt   --xml-dir ./fixtures --modes tls
netscraper --targets targets_example_ssh.txt   --xml-dir ./fixtures --modes ssh
netscraper --targets targets_example_smb.txt   --xml-dir ./fixtures --modes smb
netscraper --targets targets_example_ldap.txt  --xml-dir ./fixtures --modes ldap
netscraper --targets targets_example_rpc.txt   --xml-dir ./fixtures --modes rpc
netscraper --targets targets_example_dns.txt   --xml-dir ./fixtures --modes dns
netscraper --targets targets_example_mssql.txt --xml-dir ./fixtures --modes mssql
netscraper --targets targets_example_snmp.txt  --xml-dir ./fixtures --modes snmp
netscraper --targets targets_example_ftp.txt   --xml-dir ./fixtures --modes ftp
netscraper --targets targets_example_http.txt  --xml-dir ./fixtures --modes http

--xml-dir looks for pre-captured evidence instead of invoking the real tools (per check: <host>.xml/.txt/_chain.txt for TLS, <host>_ssh. json/.txt for SSH, <host>_smb_nmap.txt/_smbclient.txt for SMB, <host>_ldap_anon.txt/_ldap_signing.txt for LDAP, <host>_rpcdump.txt for RPC, <host>_dig.txt for DNS, <host>_mssql_nmap.txt for MSSQL, <host>_onesixtyone.txt for SNMP, <host>_ftp.txt for FTP, and <host>_http_version.txt/_http_hostheader.txt/_http_trace.txt for HTTP). fixtures/ contains real captured output from purpose-built local test servers — a genuinely self-signed cert, a genuinely incomplete chain, real weak vs. hardened SSH/SMB/LDAP/SNMP/FTP/HTTP configurations, etc. Demo target lists are split by check because each fixture host is illustrative of one specific check; a real targets file lists every host once and lets port-gating sort out which checks actually apply to it (that's the whole point of vuln-first scanning).

Output layout

Everything below lives under this run's unique folder (e.g. ~/Netscraper/results/scan_20260907_143022_a1b2c3/, or directly under --output if you passed --no-run-id):

scan_<timestamp>_<id>/
  SSL_Scan/       Expired_Certificate/ Deprecated_TLS_Protocols/ Weak_Ciphers/
                  Weak_Key_Strength/ Hostname_Mismatch/ Untrusted_Self_Signed/
                  Incomplete_Certificate_Chain/ Secure_POC/
  SSH/            Weak_Deprecated_Crypto/ Banner_Disclosure/ Secure_POC/
  SMB/            SMBv1_Enabled/ SMB_Signing_Not_Enforced/
                  Null_Session_Anonymous_Access/
                  Needs_Manual_Verification/SMB_Signing/ Secure_POC/
  LDAP/           LDAP_Anonymous_Bind_Enabled/ LDAP_Signing_Not_Enforced/
                  Needs_Manual_Verification/LDAP_Anonymous_Bind/
                  Needs_Manual_Verification/LDAP_Signing/ Secure_POC/
  RPC/            Exposed_MSRPC_Endpoint_Mapper/ MS-SCMR_atsvc_Exposed/
                  Needs_Manual_Verification/RPC_Endpoint_Mapper/ Secure_POC/
  DNS/            DNS_Recursion_Enabled/
                  Needs_Manual_Verification/DNS_Recursion/ Secure_POC/
  MSSQL/          MSSQL_Service_Information_Disclosure/ Secure_POC/
  SNMP/           Weak_Default_SNMP_Community/ Secure_POC/
  FTP/            Anonymous_FTP_Access/
                  Needs_Manual_Verification/Anonymous_FTP/ Secure_POC/
  HTTP/           Server_Version_Disclosure/ Host_Header_Injection/
                  Cross-Site_Tracing_XST/ Secure_POC/
  Dirsearch/      Found_Paths/ Secure_POC/     (no .png here -- .txt evidence only)
  CORS/           CORS_Misconfiguration/ Secure_POC/   (no .png here -- .txt evidence only)
  raw/<check>/<host>_<artifact>.txt    <- always saved, every check, every host
  summary.csv                           <- one row per (host, check)
  PT_Scan_Report.xlsx                   <- one row per finding, styled report

Each finding folder holds <host>.png (the POC screenshot) and <host>.txt (the plain-text evidence). summary.csv columns are host, port, check, status, findings — status is one of VULNERABLE, SAFE, N/A (check's required port wasn't open on this host), ERROR (the tool itself failed), or MANUAL_VERIFICATION.

PT_Scan_Report.xlsx

A styled Excel report, written alongside summary.csv at the end of every run (partial results are still written if you Ctrl-C mid-scan — see below). Columns are S.No, Vulnerability, Host, Status — a blue bold header, one row per row that's actually a real Vulnerable/Not-Vulnerable verdict:

• One row per confirmed finding, Vulnerability = the exact finding name (e.g. "Expired SSL Certificate"), Status = Vulnerable (red). A host with 3 findings from one check gets 3 separate rows.
• One row per (host, check) that came back completely clean, Vulnerability = "<CHECK> -- No Vulnerabilities Found", Status = Not Vulnerable (green).
• N/A (port not open), ERROR (tool failure), and NEEDS MANUAL VERIFICATION results are deliberately left out of this file — none of them is a real binary Vulnerable/Not-Vulnerable verdict, and forcing one in would misrepresent an inconclusive result as a finding. They're still fully recorded in summary.csv and in their own POC/evidence files regardless.

Project layout (for extending with new checks)

pyproject.toml            — packaging metadata (this is what `pip install netscraper` builds from)
LICENSE / MANIFEST.in
src/netscraper/
  __init__.py              — package version
  cli.py                   — orchestration: port-gate each family, route results to folders
  wordlists/               — small bundled wordlists (currently: SNMP default communities), shipped as package data
  core/
    targets.py             — expands IPs / hostnames / CIDR subnets into targets
    port_gate.py            — fast parallel TCP "is this port open" pre-check
    terminal_capture.py     — pty-based real ANSI output capture (reused wherever a tool colorizes)
    poc_renderer.py         — renders any (raw text, Finding, real command) as a terminal-card PNG
    concurrency.py          — auto-sizes the parallel scan worker pool from this machine's RAM/CPU
    scan_worker.py          — picklable per-(family, host) unit of work run inside a pool worker process
    rules.py                — Finding dataclass + the TLS/cert-config rule set
    families.py             — one run_xxx_family() per family; cli.py stays generic
    sslscan_runner.py / sslscan_parser.py                              — TLS family, sslscan half
    cert_chain_runner.py / cert_chain_parser.py / cert_chain_rules.py  — TLS family, chain-of-trust half
    ssh_runner.py / ssh_parser.py / ssh_rules.py                       — SSH family
    smb_runner.py / smb_parser.py / smb_rules.py                       — SMB family
    ldap_runner.py / ldap_parser.py / ldap_rules.py                    — LDAP family
    rpc_runner.py / rpc_parser.py / rpc_rules.py                       — RPC family
    dns_runner.py / dns_parser.py / dns_rules.py                       — DNS family
    mssql_runner.py / mssql_parser.py / mssql_rules.py                 — MSSQL family
    snmp_runner.py / snmp_parser.py / snmp_rules.py                    — SNMP family
    ftp_runner.py / ftp_parser.py / ftp_rules.py                       — FTP family
    http_runner.py / http_parser.py / http_rules.py                    — HTTP family
    dirsearch_runner.py / dirsearch_parser.py / dirsearch_rules.py     — DIRSEARCH family (no fixed port; render_poc=False)
    cors_runner.py / cors_parser.py / cors_rules.py                    — CORS family (no fixed port; render_poc=False)
fixtures/                 — captured evidence for offline/replay testing (dev only, not shipped in the package)

Adding a new family means: a runner (gathers evidence, live + replay), a parser (raw → plain struct), a rules module (struct → Findings, reusing core.rules.Finding, returning (confirmed_findings, manual_verification_ findings)), and one run_xxx_family() in families.py registered in FAMILIES with its required port (a plain int for a fixed TCP port, None to gate on each target's own .port, "udp:<port>" for a UDP-only service with no TCP pre-gate, or "prompt" for a family with no fixed port at all — see DIRSEARCH/CORS and cli.py's _resolve_prompt_gated_ports()). A family entry can also set "render_poc": False to skip POC screenshots entirely and save only the .txt evidence (again, DIRSEARCH). port_gate.py, terminal_capture.py, poc_renderer.py, and all of cli.py's orchestration are already generic and reusable as-is.