Skip to main content

Fast concurrent proxy scraper and checker — by BotMan Army

Project description

Proxy Hunter

Fast concurrent proxy scraper and checker

By BotMan Army | Developer: morose1337

Telegram: t.me/botmanarmy | t.me/morose1337

Install

pip install proxy-hunter

Usage

Get working proxies (scrape + check)

from proxy_hunter import get_proxies

# Returns list of working proxies
proxies = get_proxies()
print(proxies)
# ['103.152.112.162:80', '47.88.31.226:8080', ...]

# With options
proxies = get_proxies(
    workers=500,
    timeout=8,
    verbose=True,
    save_to="working.txt",
)

Only scrape (no checking)

from proxy_hunter import grab_proxies

all_proxies = grab_proxies()
print(f"Grabbed {len(all_proxies)} proxies")

Check your own proxy list

from proxy_hunter import check

my_proxies = ["103.152.112.162:80", "47.88.31.226:8080"]
result = check(my_proxies, workers=100, timeout=5, verbose=True)

print(result["working"])       # list of working proxies
print(result["working_count"]) # count
print(result["elapsed"])       # seconds
print(result["rate"])          # proxies/sec

With callback

from proxy_hunter import get_proxies

def on_hit(proxy, latency_ms):
    print(f"Found: {proxy} ({latency_ms}ms)")

proxies = get_proxies(callback=on_hit)

CLI

# Default
proxy-hunter

# Custom
proxy-hunter -w 1000 -t 4 -o results.txt --insecure -v

# As module
python -m proxy_hunter -w 500 -v

API

get_proxies(**kwargs) -> list[str]

Scrape + check. Returns working proxies.

Param Default Description
workers 800 Concurrent threads
timeout 6 Seconds per proxy
retries 2 Retry attempts
verify_ssl False Verify SSL certs
verbose False Print live output
save_to None Save to file
callback None fn(proxy, latency_ms)

grab_proxies(verbose=False) -> list[str]

Scrape only, no checking.

check(proxies, **kwargs) -> dict

Check a given list. Returns dict with working, total, tested, working_count, failed_count, elapsed, rate.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

proxy_hunter_botman-1.0.0.tar.gz (7.8 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

proxy_hunter_botman-1.0.0-py3-none-any.whl (8.7 kB view details)

Uploaded Python 3

File details

Details for the file proxy_hunter_botman-1.0.0.tar.gz.

File metadata

  • Download URL: proxy_hunter_botman-1.0.0.tar.gz
  • Upload date:
  • Size: 7.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.9

File hashes

Hashes for proxy_hunter_botman-1.0.0.tar.gz
Algorithm Hash digest
SHA256 0d77c93ce6ce7cb45693f4c93752fe2e14a03e91058f6011fbf5220a877e4fa0
MD5 41b843bf473e6b9ead429f555f5c43ef
BLAKE2b-256 9f715ea807119f61c42e9a8f83304b6a99bcec95d8cabb5f022881893d062844

See more details on using hashes here.

File details

Details for the file proxy_hunter_botman-1.0.0-py3-none-any.whl.

File metadata

File hashes

Hashes for proxy_hunter_botman-1.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 f9f29cfeb393a7e6d1416441312b0bddced746e771642b380fc0cdf1a8d9013b
MD5 ff98c397a4a3acff3595d00ca71eb67f
BLAKE2b-256 74a5eddc6ee808f71dc246149e2404a728e7c9fe0379dd18af883e7516e50067

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page