Spyglass-Pro

One dashboard. Three recon engines. Zero packets sent to your target.

Quick Start · Features · Architecture · Configuration · API · Changes · Contributing · Security · Audit

Why Spyglass-Pro?

Most OSINT tooling is a pile of scripts that assume you already know what to run, in what order, and how to read the output. Spyglass-Pro is a single dashboard that answers a different question: what is actually knowable about a target, and how do I know the answer is real?

• Three engines, one interface. Spyglass-OSINT, a stealth social harness, and Lens-Extender are merged behind a single job pool that runs them in parallel and correlates the entities they each find.
• Passive by construction, not by policy. Every probe that touched a target was deleted, not disabled — no flag, env var or config path brings it back. The audit lists every one so it can't quietly reappear.
• It refuses to lie to you. A follower list behind a login is reported as walled with a reason, never as an empty list. A recon leg that can't run is named in /api/status. A partial result never renders like a complete one.
• Honest degradation is a feature, not a bug report. Run it bare and it tells you exactly which panels are breach-only instead of quietly showing you nothing.

Built for people who need the truth. OSINT tools are dual-use. Only investigate systems and accounts you own or have explicit permission to test. See Responsible use.

Table of Contents

• What's new in 1.1
• Quick Start
• Features
• Architecture
• Installation
• Optional CLI engines
• Passive-only
• Configuration
• API
• Cases
• Testing
• Platforms
• Project structure
• Contributing
• Changelog
• Responsible use
• License

What's new in 1.1

1.1 is a hardening release: three new keyless social sources, a security pass over the seams between versions, and honest answers where 1.0 was thin. The full list is in the changelog.

• Keybase joins the social panel — keyless, and the strongest correlation source in the tree: it returns the account's cryptographically-proven identities (Twitter, GitHub, Reddit, Hacker News, DNS, web), claimed by the user's own key. That is a fact, not a guess from a shared handle.
• GitHub and GitLab are keyless too. Both report the budget tier their response actually came back with (GitHub 60 req/h, GitLab 500 req/min unauthenticated); set GITHUB_TOKEN / GITLAB_TOKEN to raise it, and the panel says which tier it got instead of assuming.
• The case store keeps its promise. With SPYGLASS_STORE_KEY set, the entity index is now encrypted alongside the result snapshot — before, the index held the same domains, IPs and emails in cleartext, leaking exactly what the encryption existed to protect.
• The default install is closed off. With no token, the dashboard answers only loopback Hosts (DNS-rebinding defense), refuses cross-site / foreign-Origin browser requests, masks proxy credentials in every response, and sets a Content-Security-Policy plus the usual framing and sniffing headers.
• IPv6 works. The ip module accepted an IPv6 literal and then always failed to resolve it; it resolves now, and a malformed address is refused instead of accepted.
• Real phone formats pass, flag-shaped input does not — +1 415 555 0100 and (415) 555-0100 are accepted; a leading - is not.
• Windows and CI. Broader Windows tool fallbacks (winget/choco), a Windows test job, and a mypy job that actually runs in CI.

Quick Start

Copy-paste ready. No API keys, no accounts, no configuration.

pip install spyglass-pro
spyglass-pro

Or from source:

git clone https://github.com/Ahmad170412/Spyglass-PRO.git && cd Spyglass-PRO
python3 -m venv .venv && source .venv/bin/activate
pip install -e .

python run.py

Open http://127.0.0.1:5000.

That's it — the keyless panels (Website, IP Address, Full Investigation, and the social adapters GitLab, GitHub, Keybase, Mastodon, Bluesky) work immediately. The email, username and phone panels need optional CLI engines. The one-line installer ships with a source checkout, not the wheel, so use it from the clone above:

python scripts/install_tools.py --check    # see what's missing
python scripts/install_tools.py            # install the missing ones

If you installed from PyPI (no checkout), install the engines directly — the installer's docstring lists the exact command per tool and platform. The Python ones go through pipx (isolated venvs); blackbird is a git clone and phoneinfoga a release binary, so the script is the reliable path when you have the source.

run.py adds the venv, pipx and blackbird locations to PATH itself, so the server finds your engines even when launched from cron, a container or a launch agent instead of a login shell.

Features

Architecture

graph LR
    UI[Dashboard<br/>Flask + vanilla JS] --> AUTH[[Auth gate<br/>Bearer token]]
    AUTH --> POOL[Job pool<br/>queue cap · 429 · timeouts]
    POOL --> SG[spyglass_osint<br/>email · username · phone<br/>website · ip · darkweb<br/>opsec · metadata]
    POOL --> LENS[lens_extender<br/>IP / domain recon]
    POOL --> SOC[social harness<br/>registry-driven adapters]
    SG --> STORE[(Case store<br/>Fernet at rest)]
    LENS --> STORE
    SOC --> STORE
    SG -. shells out to .-> CLI[holehe · sherlock<br/>maigret · ignorant · blackbird]
    UI --> EXT[(JSON / CSV / MD<br/>export artifacts)]

    style POOL fill:#1e293b,stroke:#6366f1,color:#e2e8f0
    style STORE fill:#1e293b,stroke:#06b6d4,color:#e2e8f0
    style UI fill:#1e293b,stroke:#8b5cf6,color:#e2e8f0

Every box under POOL only ever talks to third-party services. There is no edge from any of them to the target.

Installation

Requirements: Python 3.10+ (tested on 3.10–3.13 in CI), macOS, Linux or Windows.

git clone https://github.com/Ahmad170412/Spyglass-PRO.git && cd Spyglass-PRO
python3 -m venv .venv && source .venv/bin/activate
pip install -e .

Then:

python run.py                 # serves on 127.0.0.1:5000 (waitress; Flask dev server only with --debug)
python -m spyglass_pro        # equivalent
spyglass-pro                  # console entry point

pip install cloakbrowser

Optional CLI engines

The email, username and phone modules are orchestrators: they shell out to external CLI engines rather than collecting themselves.

python scripts/install_tools.py --check    # report which are missing
python scripts/install_tools.py            # install the missing ones

This covers sherlock, maigret, holehe, ignorant, blackbird, user-scanner, shodan, phoneinfoga, exiftool, dig/whois/curl and torsocks — every tool the dashboard can detect. It ships with a source checkout (git clone / pip install -e .), not the published wheel; a PyPI install has no scripts/ directory, so install the engines directly there. The script is idempotent — re-running it only installs what is still missing. Python CLIs install via pipx (isolated venvs, no dependency conflicts); system tools via brew or apt.

What you get without them

Degrading without an error is not the same as being useful, so here is exactly what a bare install can and cannot do:

Breach checking (LeakCheck + Scylla) needs no account and no CLI tool, so it works on a bare install. Website and IP Address run without any recon engine, but their richer legs depend on tools that may be absent — Website uses dig, whois and curl; IP uses shodan for host/port data. Full Investigation merges the modules above, so its coverage is theirs. GET /api/status reports exactly which tools are missing on your machine.

To confirm what the running server can see, GET /api/status returns a per-tool true/false map, a missing_tools list and a per-panel degraded map.

holehe and ignorant enumerate which platforms an email or phone number is registered on. That is active account enumeration against third-party services and it sits against Spyglass-Pro's passive-only posture. Only use them on identifiers you own or have permission to test.

Passive-only

Spyglass-Pro never opens a connection to a target. Every query goes to a third party: DNS resolvers, Certificate Transparency logs, WHOIS registries, InternetDB, RDAP, RIPEstat, Wayback, crt.sh, CertSpotter, HackerTarget and breach databases. Shodan, GreyNoise, Censys, AbuseIPDB, VirusTotal and NVD enrichment are wired but need an API key.

There is no port scan, no directory brute force, no HTTP probing of discovered hosts, no zone-transfer attempt and not even a fetch of the target's homepage. That code was removed, not disabled — the audit inventory lists every removed function, so none of it can be switched back on by configuration or a stray flag.

Where the line sits: ordinary DNS record lookups (A, MX, NS, TXT, SPF, DMARC, CNAME) are still made, because no useful host recon exists without them and every OSINT tool treats them as passive.

What that costs you, plainly: no HTTP headers, no TLS certificate or cipher details, no technology fingerprinting, no robots.txt or sitemap, no page content, no favicon hash and no exposed-file findings. Active collection lives in a separate project.

Configuration

All settings are environment variables (CLI flags exist for the basics — see spyglass-pro --help):

Non-loopback deployments must set SPYGLASS_PRO_TOKEN; the server warns at startup if you don't. While no token is set the dashboard answers only requests whose Host is loopback (or listed in SPYGLASS_PRO_ALLOWED_HOSTS) — that is what stops a DNS-rebinding page from reaching it. Serving it under a real hostname without a token therefore needs SPYGLASS_PRO_ALLOWED_HOSTS, e.g.:

SPYGLASS_PRO_TOKEN=$(openssl rand -hex 24) python3 run.py

Independently of the token, browser requests tagged cross-site (Sec-Fetch-Site: cross-site) or carrying a foreign Origin are refused, and every response sets Content-Security-Policy, X-Frame-Options, X-Content-Type-Options and Referrer-Policy.

API

Every route is under /api/* and requires the Bearer token when SPYGLASS_PRO_TOKEN is set.

# What's installed, what's missing, what degraded
curl -H "Authorization: Bearer $TOKEN" http://127.0.0.1:5000/api/status

# Run a module
curl "http://127.0.0.1:5000/api/spyglass/run?module=website&target=example.com" \
     -H "Authorization: Bearer $TOKEN"

# Correlated multi-input investigation
curl -X POST http://127.0.0.1:5000/api/investigate/run \
     -H "Authorization: Bearer $TOKEN" \
     -H "Content-Type: application/json" \
     -d '{"website":"example.com","email":"user@example.com"}'

# Stored-run queries
curl "http://127.0.0.1:5000/api/cases/diff?target=example.com&qtype=website" \
     -H "Authorization: Bearer $TOKEN"

A busy server returns HTTP 429 for a full job queue, distinct from a scan that ran and failed — so a client can tell back-pressure from a broken payload.

Cases

Every scan can be stored. Stored runs keep a full result snapshot plus an entity index (domains, IPs, emails, phones, orgs, onions, breaches), which is what the Stored Cases panel reads:

• Diff last two runs — entities that appeared or disappeared since the previous run of the same target and query type.
• Timeline — every stored run of a target, oldest first.
• Export dossier — the latest run per query type merged into one JSON profile.

Data at rest is encrypted with Fernet when SPYGLASS_STORE_KEY or SPYGLASS_STORE_PASS is set; otherwise it is stored in plaintext. This covers both the result snapshot and the entity index (domains, IPs, emails, orgs) — the index rows hold the same targets, so encrypting one without the other would leak them. JSON/CSV/report artifacts from a scan land in the working directory of the run.

Testing

.venv/bin/python tests/run_tests.py

545 offline checks — the three engines' own suites plus the merge layer (API surface, auth gate, request-origin gate, proxy validation and masking, job pool, target validation incl. IPv6, entity encryption, case queries, cross-platform filename and decoding behaviour) and the node-checked dashboard JS. No network, no browser, finishes in a few seconds.

CI runs the suite on Python 3.10–3.13 (plus macOS and Windows), type-checks with mypy, lints with ruff, node --checks the dashboard JS, and asserts the built wheel actually carries LICENSE, NOTICE and all three vendored engines.

Platforms

macOS, Linux and Windows are supported. CI runs the suite on Ubuntu (Python 3.10–3.13) and macOS; Windows ships documented fallbacks for the tools it lacks. Two platform differences are worth knowing before you file a bug:

• Windows has no dig or whois. The DNS legs fall back to Python's resolver and the HTTP legs to urllib, so the modules still run — but DNS record tables, SPF/DMARC and WHOIS come back thinner until you install the tools. GET /api/status lists exactly which are missing on your machine.
• Windows reserved filenames. CON, PRN, AUX, NUL, COM1–COM9 and LPT1–LPT9 cannot be used as filenames in any case, so Spyglass-Pro prefixes them rather than producing artifacts that fail to open there.

Project structure

run.py                     dev entry point
scripts/install_tools.py   idempotent CLI-engine installer
tests/                     545 offline checks (run_tests.py)
assets/logo.svg            README logo
spyglass_pro/
  app.py                   Flask routes + auth gate + proxy validation
  runner.py                job pool (queue cap, timeouts), engine dispatch
  social/                  multi-platform social harness (schema, registry,
                           adapters) — merge layer, not vendored
  cli.py                   `spyglass-pro` console entry point
  templates/ static/       dashboard UI
  modules/                 vendored engines (see NOTICE)

The three engines live behind a directory boundary on purpose, so the next sync from upstream is mechanical. NOTICE records exactly where each copy differs from its origin — including the bug fixes that exist only here.

Contributing

Read CONTRIBUTING.md. The short version: the passive-only rule overrides everything else, new behaviour needs a test that fails without it, and a partial result must never look like a complete one.

Responsible use

OSINT tools are dual-use. Only investigate systems and accounts you own or have explicit permission to test.

Instagram collection may violate those platforms' terms of service. Because Spyglass-Pro is passive-only it does not touch a target's network — but the social module still drives a real browser against a real account, so use burner credentials you are willing to lose. Passive queries also disclose your investigation target to third-party services and your source IP to all of them; use a proxy if that matters to you.

You are responsible for compliance with applicable laws.

License

MIT — see LICENSE. One copyright holder covers this repository in full, including the engines under spyglass_pro/modules/; NOTICE records where those came from and why they are kept behind a directory boundary.