SUBX — Subdomain Recon Framework

SUBX is a fast subdomain enumeration, DNS resolution, and HTTP probing tool built for security researchers, bug bounty hunters, and pentesters.

It pulls subdomains concurrently from 12 passive OSINT sources (Shodan, Censys, VirusTotal, crt.sh, Chaos, etc.), filters them against your target scope, stores everything in PostgreSQL or SQLite, and integrates with dnsx and httpx for DNS resolution and web probing.

How it works

                       ┌─────────────────────────────────┐
                       │               CLI               │
                       └────────────────┬────────────────┘
                                        │
                                        ▼
                       ┌─────────────────────────────────┐
                       │         Services Layer          │
                       └──────┬───────────────────┬──────┘
                              │                   │
                              ▼                   ▼
                       ┌──────────────┐   ┌──────────────┐
                       │PluginManager │   │ ToolManager  │
                       └──────┬───────┘   └──────┬───────┘
                              │                   │
                              ▼                   ▼
                       ┌──────────────┐   ┌──────────────┐
                       │ Passive APIs │   │ Active Tools │
                       │ (12 sources) │   │(dnsx & httpx)│
                       └──────────────┘   └──────────────┘

1. Passive Enumeration: Queries 12 APIs concurrently using asyncio with real-time terminal streaming.
2. Scope Filtering & Deduplication: Cleans findings, checks wildcard boundaries, and deduplicates against existing target assets.
3. DNS Probing (dnsx): Resolves subdomains to IP addresses and tracks active A/CNAME records.
4. HTTP Probing (httpx): Probes live web targets for HTTP status codes, titles, and tech stacks.
5. Smart Pipeline: When both dnsx and httpx are configured, HTTP probing automatically runs only against DNS-resolved active hosts to save time.

Features

• Concurrent Async Engine — Parallel queries across all passive OSINT sources.
• Scope Control — In-scope and out-of-scope domain boundary enforcement.
• DNS & HTTP Probing — Built-in dnsx (DNS resolution & IP mapping) and httpx (liveness, titles, technologies).
• Smart Pipeline Optimization — Runs HTTP probes exclusively against subdomains that resolve via DNS.
• Database Storage — SQLite out of the box, with full PostgreSQL support for team workflows.
• Historical Tracking — Tracks first_seen, last_seen, and last_seen_alive timestamps for asset monitoring.
• DB Query & Views — Terminal tables for web status (--web), DNS resolution (--dns), tech stack, or custom SQL.
• Project Export — Automatically generates recon/ folders with plain-text output files (subdomains.txt, alive.txt, ips.txt, techs.txt).

Passive Sources

Installation

From PyPI (Recommended)

pip install subx-recon

From Source

git clone https://github.com/RiadhBenlamine/SubX.git
cd SubX
pip install -e .

Or using uv:

git clone https://github.com/RiadhBenlamine/SubX.git
cd SubX
uv sync

Debian / Ubuntu / Kali (.deb)

Download the .deb from the latest release and install:

sudo apt install ./subx_2.1.1-1_all.deb

Requirements: Python >= 3.10

Configuration

Create a config.yaml file (a sample is available in config_samples/config.yaml_sample):

scope:
  - example.com
  - target.org

out_of_scope:
  - testing.example.com

# Omit sources to run all available plugins
sources:
  - ShodanPlugin
  - CrtshPlugin
  - ChaosPlugin

api_keys:
  SHODAN_API: "your-key-here"
  VIRUSTOTAL_API: "your-key-here"
  OTX_API: "your-key-here"
  URLSCAN_API: "your-key-here"
  CHAOS_API: "your-key-here"
  BEVIGIL_API: "your-key-here"
  VIEWDNS_API: "your-key-here"
  CENSYS_API: "your-key-here"

# Tool parameters for active probing pipeline
tools:
  dnsx:
    threads: 100
  httpx:
    threads: 50
    rate-limit: 150

You can also store API keys in ~/.config/subx/.env (e.g. SHODAN_API=your-key). config.yaml values take priority.

Usage

subx [COMMAND] [OPTIONS]

Quick Reference

subx enum -c config.yaml              # Enumerate subdomains
subx probe -d example.com             # HTTP probe
subx resolve -d example.com           # DNS resolve
subx db                               # List tracked domains
subx db -d example.com --web          # Web liveness view
subx db -d example.com --dns          # DNS resolution view
subx db -d example.com --alive -o a.txt   # Export alive subs
subx db -d example.com --tech Nginx   # Filter by technology
subx db --query "SELECT ..."          # Raw SQL query
subx export -d example.com            # Export project files
subx setup                            # Setup PostgreSQL
subx import subx.db                   # Import SQLite → PostgreSQL
subx migrate                          # Run schema migrations

Enumerate Subdomains

subx enum -c config.yaml

Runs all configured passive plugins against your scope. Results stream to your terminal in real-time and are saved to the database.

• -c, --config — Path to config file (required)
• --save / --no-save — Toggle database saving (default: save)
• -p, --project — Auto-export project directory after scan
• -D, --debug — Enable verbose debug logs

HTTP Probing (probe)

Probe stored subdomains for web liveness, HTTP codes, titles, and technologies using httpx:

subx probe -d example.com

Note: If dnsx is configured in config.yaml, probe automatically resolves subdomains first and probes only active DNS hosts.

• -d, --domain — Target domain (required)
• -o, --out <file> — Save alive subdomains to file
• -T, --out-tech <file> — Save subdomains with tech stack (e.g. app.example.com [Nginx, React])
• -p, --project — Export project directory after probing

DNS Resolution (resolve)

Resolve stored subdomains using dnsx to get IP addresses:

subx resolve -d example.com

• -d, --domain — Target domain (required)
• -r, --resolvers <file> — Custom resolvers file (one IP per line)
• -o, --out <file> — Save resolved subdomains to file
• -I, --out-ip <file> — Save resolved subdomains with IP addresses (subdomain [ip])
• -p, --project — Export project directory structure after probing

Query Database (db)

Inspect tracked assets, view web status or DNS mappings, and export filtered targets:

# List tracked target domains
subx db

# List subdomains for a target
subx db -d example.com

# Show DNS resolution view (Resolved status + IP Address)
subx db -d example.com --dns

# Show Web liveness view (Alive status, HTTP Code, Title, Tech)
subx db -d example.com --web

# Filter by alive status
subx db -d example.com --alive --web

# Filter by DNS resolution status
subx db -d example.com --resolved --dns

# Filter by technology or plugin source
subx db -d example.com --tech Nginx
subx db -d example.com --plugin ShodanPlugin

# Run custom read-only SQL query
subx db --query "SELECT subdomain, ip, status_code FROM subx_subdomain WHERE target='example.com' AND alive=true"

# Export alive subdomains to file
subx db -d example.com --alive -o live.txt

# Delete domain records
subx db -d example.com --delete

Export Project Workspace

subx export -d example.com

Creates a structured output directory under projects/:

projects/
  └── example.com/
        └── recon/
              ├── subdomains.txt
              ├── alive.txt
              ├── dead.txt
              ├── techs.txt
              ├── status.txt
              ├── ips.txt
              └── sources.txt

You can also pass -p / --project to enum, probe, resolve, or db commands.

Database Setup & Migrations

# Setup PostgreSQL database connection
subx setup -H 127.0.0.1 -u postgres -W "password" -n subx

# Import an existing SQLite database into PostgreSQL
subx import subx.db

# Apply schema migrations when updating SUBX
subx migrate

PostgreSQL Setup

SUBX uses SQLite by default. To connect to a PostgreSQL database:

1. Run subx setup (saves connection info to ~/.config/subx/config.yaml), or
2. Set the DATABASE_URL environment variable:

export DATABASE_URL="postgresql+asyncpg://postgres:password@localhost:5432/subx"

Release Automation

python scripts/release.py patch  # 2.1.0 -> 2.1.1
python scripts/release.py minor  # 2.1.0 -> 2.2.0

Contributing

See DEV_DOCS.md for technical architecture details, plugin development guides, and implementation rules.

License

MIT — see LICENSE for details.