osint-darkweb-pkg 0.1.0

A Darkweb OSINT tool for scraping and searching onion sites.

Darkweb OSINT Package

A modular Open Source Intelligence (OSINT) tool designed to search and scrape data from the Darkweb (Tor network). This package aggregates results from multiple darkweb search engines and provides a concurrent scraper to extract content from .onion sites.

Features

• Multi-Engine Search: Queries 15+ darkweb search engines (Ahmia, Torch, OnionLand, etc.) concurrently.
• Concurrent Scraping: Rapidly extracts text content from multiple .onion URLs simultaneously.
• Tor Integration: Built-in SOCKS5 proxy configuration for seamless routing through the Tor network.
• User-Agent Rotation: Randomizes user agents to minimize blocking.

Prerequisites

Crucial: You must have the Tor service running on your machine for this package to function. This package routes traffic through the local Tor SOCKS5 proxy (default port 9050).

• Linux (Debian/Ubuntu):

  sudo apt install tor
  sudo service tor start
  

• Windows: Download the Tor Browser or the Tor Expert Bundle and ensure it is running.

Installation

This package is managed via pyproject.toml. You can install it directly:

pip install osint-darkweb.pkg

Usage

** 1. Searching** Search for a keyword across multiple hidden service search engines.

from osint_darkweb_pkg import get_search_results

query = "example query"

Returns a list of dictionaries: [{'title': '...', 'link': '...'}]

results = get_search_results(query)

print(f"Found {len(results)} links.")

** 2. Scraping** Extract text content from a list of .onion URLs. from osint_darkweb_pkg import scrape_multiple

specific .onion links or results from the search step

urls = [ {'link': 'http://example1.onion', 'title': 'Example 1'}, {'link': 'http://example2.onion', 'title': 'Example 2'} ]

Returns a dictionary: {'http://example1.onion': 'Page content...'}

data = scrape_multiple(urls)

for url, content in data.items(): print(f"URL: {url}\nContent: {content[:100]}...\n")

Structure

search.py: Handles sending queries to search engine endpoints and parsing results.

scrape.py: Handles visiting specific .onion URLs and extracting text.

utils.py: Manages shared resources like Tor proxy configuration and User-Agent rotation.

Disclaimer

This tool is intended for educational and research purposes only (e.g., security research, OSINT investigations). The authors are not responsible for any misuse of this tool. Accessing the dark web may be illegal or monitored in certain jurisdictions. Use responsibly.