headless-driver

Lightweight Python package to manage Selenium WebDriver in headless mode with proxy support, stealth tweaks, auto-driver installation, multi-driver management, download handling, and advanced search-scraping utilities

Features

• Headless and non-headless Chrome management
• Temporary or persistent user-data directories (profiles)
• HTTP and SOCKS proxy support
• Optional stealth mode (integrates with selenium-stealth when available)
• Automatic ChromeDriver install (via webdriver-manager) as an optional dependency
• Download folder management and automatic cleanup
• Screenshot and PDF export via Chrome DevTools
• Multi-driver manager to run many isolated browser instances
• Advanced search scraper (DuckDuckGo default) with title, snippet, favicon, cached link, and batch search/export support

Installation

pip install headless-driver

Usage

from headless import Headless

hl = Headless()
driver = hl.get_driver()
driver.get("https://example.com")
print(driver.title)
hl.quit()

Or use as a context manager:

from headless import Headless

with Headless() as driver:
    driver.get("https://example.com")
    print(driver.title)

Search

from headless import SearchScraper, Headless

hl = Headless()
driver = hl.get_driver()

scraper = SearchScraper(driver=driver, max_results=5)

results = scraper.search("Nuhman PK github")
print(results)

Stealth

from headless import ExtendedHeadless

hl = ExtendedHeadless(stealth=True)

driver = hl.get_driver()
driver.get("https://example.com")

print(driver.title)

hl.quit()

Proxy

from headless import ExtendedHeadless

hl = ExtendedHeadless(proxy="socks5://127.0.0.1:9050")

driver = hl.get_driver()
driver.get("https://example.com")

print(driver.title)

hl.quit()

Take Screen / Export PDF

from headless import ExtendedHeadless

hl = ExtendedHeadless(download_dir="/tmp/hd_downloads")
d = hl.get_driver()

d.get("https://example.com")

hl.screenshot("/tmp/example.png")
hl.save_pdf("/tmp/example.pdf")

hl.quit()

Auto Install Driver

from headless import ExtendedHeadless

hl = ExtendedHeadless(auto_install=True)
d = hl.get_driver()

d.get("https://example.com")

hl.screenshot("/tmp/example.png")
hl.save_pdf("/tmp/example.pdf")

hl.quit()

Multi driver manager

from headless import MultiDriverManager

mgr = MultiDriverManager()
a = mgr.create("bot1", stealth=True, auto_install=True)
b = mgr.create("bot2", proxy="http://1.2.3.4:3128", download_dir="/tmp/d2", auto_install=True)
da = a.get_driver()
db = b.get_driver()
mgr.quit_all()

Advanced scraper

from headless import AdvancedSearchScraper

# search_engine: "duckduckgo" (default), "bing" or "google"
scr = AdvancedSearchScraper(headless_options={"headless": True}, max_results=5)

res = scr.search("python headless")

batch = scr.search_batch(["python headless", "selenium stealth"], max_workers=2)

scr.export("results.json")   # .json and .csv are supported
scr.quit()

Search engines defend aggressively against automation. When one serves a bot check instead of results, search() reports the block and returns an empty list; Google does this to headless browsers on most networks.

API Documentation

Headless class

Headless(
    user_data_dir: Optional[str] = None,
    window_size: Tuple[int, int] = (1920, 1080),
    user_agent: Optional[str] = None,
    headless: bool = True,
    chrome_driver_path: Optional[str] = None,
    additional_args: Optional[List[str]] = None,
    remote_url: Optional[str] = None,
    verbose: bool = False,
)

• user_data_dir: Path for Chrome user data (temporary if not provided)
• window_size: Browser window size (default: 1920x1080)
• user_agent: Custom user agent string
• headless: Run Chrome in headless mode (default: True)
• chrome_driver_path: Path to chromedriver executable. When omitted, one is auto-detected on PATH and common install locations; if that driver turns out to mismatch the installed Chrome, a matching one is downloaded automatically
• additional_args: List of extra Chrome arguments
• remote_url: Use remote Selenium server if provided
• verbose: Print driver setup and teardown diagnostics

Methods

• get_driver(): Returns a Selenium WebDriver instance
• quit(): Quits the driver and cleans up user data

Install Driver on Linux

sudo apt update
sudo apt install -y chromium chromium-driver

Happy coding 🚀