osintengine 1.0.8

The OSINT investigation engine — 7-phase pipeline, persistent knowledge graph, community intelligence

🕵️ Watson — The OSINT Investigation Engine

Stop searching. Start investigating.

---

Watson doesn't just look things up. It investigates.

Most tools give you a search bar. Watson gives you a forensics lab. It runs a 7-phase intelligence pipeline across 16+ data sources, cross-references every finding, builds a persistent knowledge graph that remembers across cases, and produces a structured intelligence brief — not a list of links.

One target. One command. A dossier that would take an analyst hours.

Why I built this. OSINT tools fall into two camps: shallow search engines that return links, and expensive enterprise platforms locked behind sales calls. There's no open, production-grade investigation engine that chains 16+ data sources, cross-references findings, remembers across cases, and produces real intelligence briefs — not just lists of URLs. Watson fills that gap. It's free, open-source, model-agnostic, and runs on your machine. No cloud dependency. No vendor lock-in. Just answers.

---

The difference between searching and investigating

You type...	Most tools return...	Watson returns...
"John Chen"	Wikipedia link, maybe a news article	Employment history, board positions, corporate registrations, domain ownerships, sanctions screening, adverse media, timeline of professional milestones, entity relationship graph
"example.com"	WHOIS record	Domain history, SSL certificates, subdomains, DNS records, IP geolocation, hosting provider, Wayback snapshots, linked social accounts, VirusTotal reputation
"crypto-wallet-0x..."	Nothing useful	Transaction graph, exchange links, associated addresses, risk scoring, dark web mentions

Watson doesn't just find what you asked for. It finds what you didn't know to ask.

---

Quick Start

# pip
pip install osintengine

# or uv (recommended)
uv pip install osintengine

# First-time setup (60 seconds)
watson onboard

# Start investigating
watson web

Open http://localhost:8777. Type a name. Watch the investigation unfold in real time.

No API key? Watson's free tier works immediately — Wikidata, crt.sh, Wayback Machine, DuckDuckGo, ICIJ Offshore Leaks, and more run without any keys. Add an LLM key for intelligence synthesis (DeepSeek offers a free tier).

---

How It Works

The 7-Phase Pipeline

Watson doesn't guess. It follows a deterministic methodology refined across hundreds of real cases:

  investigate("target")
         │
         ▼
  ┌──────────────────────────────────────┐
  │ PHASE 1 · CLASSIFY                   │
  │ Person? Company? Domain? Wallet?     │
  │ Checks knowledge graph for priors    │
  │ Selects investigation strategy       │
  └──────────────┬───────────────────────┘
                 ▼
  ┌──────────────────────────────────────┐
  │ PHASE 2 · SURFACE                    │
  │ Wikipedia, Wikidata, DDG dorking     │
  │ crt.sh certificates, Wayback history │
  │ WHOIS, DNS, SSL, URLscan             │
  │ Social media presence, news mentions │
  └──────────────┬───────────────────────┘
                 ▼
  ┌──────────────────────────────────────┐
  │ PHASE 3 · PIVOT                      │
  │ Email → accounts across platforms    │
  │ Username → 300+ site presence check  │
  │ Breach data, credential exposure     │
  │ Identifier chaining — Bazzell method │
  └──────────────┬───────────────────────┘
                 ▼
  ┌──────────────────────────────────────┐
  │ PHASE 4 · DEEP                       │
  │ OpenSanctions, Interpol, OFAC        │
  │ OpenCorporates, SEC EDGAR            │
  │ ICIJ Offshore Leaks, OCCRP Aleph     │
  │ Court records, financial registries  │
  │ VirusTotal domain/IP reputation      │
  └──────────────┬───────────────────────┘
                 ▼
  ┌──────────────────────────────────────┐
  │ PHASE 5 · DARK (auto-escalated)      │
  │ Ransomware checks, dark web mentions │
  │ Triggered by criminal/financial signals│
  │ Skipped for clean targets            │
  └──────────────┬───────────────────────┘
                 ▼
  ┌──────────────────────────────────────┐
  │ PHASE 6 · ANALYZE                    │
  │ Cross-reference all phases           │
  │ Entity resolution & deduplication    │
  │ Identity correlation scoring         │
  │ Source tiering: PRIMARY→UNVERIFIED   │
  │ LLM synthesis → structured brief     │
  └──────────────┬───────────────────────┘
                 ▼
  ┌──────────────────────────────────────┐
  │ PHASE 7 · REPORT                     │
  │ Markdown dossier saved to disk       │
  │ Exec summary + risk themes           │
  │ Timeline of key events               │
  │ Evidence gaps flagged                │
  │ Entities indexed in knowledge graph  │
  │ Opt-in: publish to community graph   │
  └──────────────────────────────────────┘

Every finding is verified

Watson doesn't trust anything blindly. Each finding gets:

• Source tier: PRIMARY (court records, sanctions) → SECONDARY (news, corporate registries) → TERTIARY (Wikipedia, social media) → UNVERIFIED
• HTTP verification: Every URL is reachability-checked
• Confidence scoring: Weighted by source authority and corroboration across independent sources
• Chain of custody: Every finding links back to exactly where it came from

---

What Makes Watson Different

🧠 Cross-case memory

Watson remembers. Case #47 automatically surfaces connections from Case #12. Investigate a person who appeared in a previous company's due diligence? Watson flags it before you even ask.

$ watson investigate "John Smith"
⚠️  Prior findings: This person appeared in Case #12 (Acme Corp due diligence, 3 months ago)
    — Board member at 2 companies
    — Mentioned in OFAC advisory
    — Already verified: email [e-mail]
    Continuing with fresh investigation...

🌐 Community knowledge graph

Watson ships with an MCP server. Opt-in to publish your findings (per-case consent, never automatic) and search across all published investigations:

watson_search("John Smith")  →  3 cases, 12 entities, 8 connections
watson_traverse("Acme Corp") →  Board → Subsidiaries → Adverse media
watson_context("example.com") →  2 prior investigations of this domain

Run your own instance or connect to a community graph. Collective intelligence without sacrificing privacy.

🔌 Model and agent agnostic

Watson doesn't lock you in. Bring your own LLM — any OpenAI-compatible API works:

# DeepSeek (free tier available)
export WATSON_API_KEY=sk-...
export WATSON_MODEL=deepseek-chat

# Anthropic Claude
export WATSON_API_KEY=sk-ant-...
export WATSON_MODEL=claude-sonnet-4
export WATSON_API_BASE=https://api.anthropic.com/v1

# Or any provider: OpenAI, Groq, Gemini, local Ollama...

Watson auto-detects available agent runtimes (Hermes, Direct) and appears in onboarding. Add a new adapter in ~80 lines — it just works.

---

Investigation Modes

Mode	Duration	What it does	When to use
background_check	30–60s	Classify + Surface	Quick identity/domain verification
due_diligence	2–5 min	+ Pivot + Deep	Business verification, adverse media, KYC
deep_investigation	5–15 min	All 7 phases	Full dossier, criminal/legal, dark web
twin_connection	3–8 min	Dedicated relational pipeline	Find hidden connections between two targets

---

Data Sources

Free (no keys required)

crt.sh · URLscan.io · Wayback Machine · DuckDuckGo · Wikipedia · Wikidata · ICIJ Offshore Leaks · OCCRP Aleph · BuiltWith · Instant Username Search · OpenSky Network · FlightAware

Optional (individually configurable)

Service	What it does	Cost
OpenSanctions	Sanctions, PEP, entities	$0–50/mo
OpenCorporates	Global company registries	~$50/mo
VirusTotal	Domain/IP reputation	$0–50/mo
HIBP	Breach/credential exposure	$4/mo

All paid APIs are optional. Watson is fully functional on free sources alone.

---

Project Structure

watson-osint/
├── src/watson/                    # Core investigation engine
│   ├── orchestration/
│   │   ├── engine.py              # 7-phase pipeline (5,500+ lines)
│   │   ├── synthesis.py           # LLM report generation
│   │   ├── resolution.py          # Entity resolution & dedup
│   │   └── target_profile.py      # Target classification
│   ├── tools/                     # 17 specialized investigation tools
│   │   ├── people.py              # Person lookup, HIBP, breach data
│   │   ├── corporate.py           # Company registries, sanctions
│   │   ├── websites.py            # Domain, WHOIS, SSL, subdomain
│   │   ├── blockchain.py          # Wallet analysis, exchange links
│   │   ├── darkweb.py             # Ransomware, dark web indicators
│   │   ├── social_media.py        # Social presence, account linking
│   │   └── ...                    # Shodan, MarineTraffic, satellite, etc.
│   ├── graph/                     # Knowledge graph engine
│   └── infra/                     # Caching, rate limiting, retry logic
├── watson/                        # Web app, CLI, toolkit
│   ├── web/app.py                 # FastAPI application (port 8777)
│   ├── cli.py                     # Terminal interface
│   ├── agents/                    # Pluggable agent adapters
│   └── mcp_server.py              # Community graph MCP server
├── frontend/                      # React UI (Vite + Tailwind)
├── tests/                         # Integration & unit tests
└── deploy.sh                      # One-command production deploy

---

License

Apache License 2.0 — free for any use, private or commercial. Includes explicit patent protection for all contributors and users. Must retain copyright and license notices. No network-use trigger — deploy Watson internally, in the cloud, or embedded in your product without concern.

---

"When you have eliminated the impossible, whatever remains, however improbable, must be the truth."
— Sherlock Holmes