Skip to main content

Scraper of free online proxies

Project description

getproxies

Get some free proxies scraped from free proxy sources.

from getproxies import get_proxies

proxies = get_proxies()
print(proxies[:10])
# [http://196.18.215.153:3128, https://36.67.89.179:65205, http://35.247.192.53:3128, socks5://113.54.158.40:1080, https://180.122.51.154:9999, socks4://117.44.28.152:9201, https://178.20.137.178:43980, https://109.86.121.118:46333, https://148.77.34.194:39175, socks4://114.99.16.195:1080]

or using cli:

$ poetry run getproxies --help
Usage: getproxies [OPTIONS]

  scrape free proxies from all sources and put it to STDOUT

Options:
  -p, --protocol [https|http|socks4|socks5]
                                  restrict to specific protocol
  -c, --country TEXT              only proxies from specified country (ISO
                                  double char code, e.g. US)

  -l, --limit INTEGER             limit proxy retrieval - increases
                                  performance

  -f, --format TEXT               proxy output format, available variables: ho
                                  st,port,protocol,code,country,anonymous,sour
                                  ce  [default: {protocol}://{host}:{port}]

  --help                          Show this message and exit.

Install

$ pip install getproxies
or for latest code
$ pip install -U git+https://github.com/granitosaurus/getproxies

Sources

Currently these sources are supported

source spider
http://free-proxy-list.net FreeProxyListSpider
http://proxy-daily.com ProxyDailySpider

extended usage

For more detailed proxy query a ProxyManager can be used:

from getproxies import ProxyManager

proxies = ProxyManager(
    limit=10,  # only 10 proxies
    protocol='socks5',  # only socks5 proxies
    country='us',  # only us proxies
)

limit parameter can reduce retrieval time

Proxy objects also have extended attributes:

class Proxy:
    host: str
    port: str
    protocol: str
    code: str = ''
    country: str = ''
    anonymous: bool = False
    source: str = ''

as strings they resolve to protocol://host:port template.

Project details


Release history Release notifications

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for getproxies, version 1.0.0
Filename, size File type Python version Upload date Hashes
Filename, size getproxies-1.0.0-py3-none-any.whl (6.4 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size getproxies-1.0.0.tar.gz (4.8 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page