Specter

Specter is a highly resilient, high-performance Python library designed for robust image downloading. It implements a multi-tier fallback pipeline to address restrictive network configurations, anti-scraping protections (Cloudflare), TLS fingerprinting, and hotlinking restrictions.

Features

• Multi-Tier Fallback Pipeline: Sequentially attempts multiple download strategies (Direct, DPI-Resilient proxy, Browser-rendering, and Cache-proxy) to ensure successful retrieval.
• DPI Resilience via ClientHello Segmentation: Implements a local loopback proxy that segments the TLS ClientHello packet, helping resolve network connection limitations.
• Domain Strategy Caching: Thread-safe caching mechanism that stores the successful strategy for each domain, skipping slow timeouts and retries on subsequent requests.
• Thread-Local Connection Pooling: Reuses TCP and TLS connections via Keep-Alive sessions per thread to achieve sub-100ms request times.
• Parallel Batch Downloading: Supports concurrent downloading of multiple images using a built-in ThreadPoolExecutor.
• Headless Browser Fallback: Integrates headless Chromium rendering (via DrissionPage) as a fallback to extract images from canvas elements or screenshots.

Installation

Standard Installation

pip install NullGazeX

Full Installation (with Headless Browser support)

pip install NullGazeX[full]

Quick Start

1. Download a Single Image

from specter import download_image

# Download an image with automatic header and referrer matching
download_image(
    url="https://example.com/images/photo.jpg",
    output_path="outputs/photo.jpg",
    verbose=True
)

2. Concurrent Batch Downloading

To download multiple images concurrently at maximum speed, use the batch downloading interface:

from specter import download_images

# Define a list of targets (url, output_path)
targets = [
    ("https://example.com/img1.jpg", "outputs/img1.jpg"),
    ("https://example.com/img2.jpg", "outputs/img2.jpg"),
    ("https://example.com/img3.jpg", "outputs/img3.jpg")
]

# Run concurrent downloads (default max_workers is 20)
results = download_images(targets, max_workers=10, verbose=True)

for i, res in enumerate(results):
    if isinstance(res, Exception):
        print(f"Failed to download image {i+1}: {res}")
    else:
        print(f"Successfully saved to: {res}")

3. Advanced Custom Configuration

You can customize headers or use the object-oriented API directly:

from specter import ImageDownloader

downloader = ImageDownloader(verbose=True)

custom_headers = {
    "Referer": "https://custom-referer.com/",
    "Cookie": "session_token=xyz123"
}

# Download using the custom downloader instance
downloader.download(
    url="https://example.com/protected/image.png",
    output_path="outputs/image.png",
    headers=custom_headers
)

Technical Specifications & Strategies

License

This project is licensed under the MIT License. See LICENSE for details.