Skip to main content

Scraper of free online proxies

Project description

getproxies

Get some free proxies scraped from free proxy sources.

from getproxies import get_proxies

proxies = get_proxies()
print(proxies[:10])
# [http://196.18.215.153:3128, https://36.67.89.179:65205, http://35.247.192.53:3128, socks5://113.54.158.40:1080, https://180.122.51.154:9999, socks4://117.44.28.152:9201, https://178.20.137.178:43980, https://109.86.121.118:46333, https://148.77.34.194:39175, socks4://114.99.16.195:1080]

or using cli:

$ poetry run getproxies --help
Usage: getproxies [OPTIONS]

  scrape free proxies from all sources and put it to STDOUT

Options:
  -p, --protocol [https|http|socks4|socks5]
                                  restrict to specific protocol
  -c, --country TEXT              only proxies from specified country (ISO
                                  double char code, e.g. US)

  -l, --limit INTEGER             limit proxy retrieval - increases
                                  performance

  -f, --format TEXT               proxy output format, available variables: ho
                                  st,port,protocol,code,country,anonymous,sour
                                  ce  [default: {protocol}://{host}:{port}]

  --help                          Show this message and exit.

Install

$ pip install getproxies
or for latest code
$ pip install -U git+https://github.com/granitosaurus/getproxies

Sources

Currently these sources are supported

source spider
http://free-proxy-list.net FreeProxyListSpider
http://proxy-daily.com ProxyDailySpider

extended usage

For more detailed proxy query a ProxyManager can be used:

from getproxies import ProxyManager

proxies = ProxyManager(
    limit=10,  # only 10 proxies
    protocol='socks5',  # only socks5 proxies
    country='us',  # only us proxies
)

limit parameter can reduce retrieval time

Proxy objects also have extended attributes:

class Proxy:
    host: str
    port: str
    protocol: str
    code: str = ''
    country: str = ''
    anonymous: bool = False
    source: str = ''

as strings they resolve to protocol://host:port template.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

getproxies-1.0.0.tar.gz (4.8 kB view hashes)

Uploaded source

Built Distribution

getproxies-1.0.0-py3-none-any.whl (6.4 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page