Skip to main content

A description of your project

Project description


title: Title

summary: "summary"

Proxy Manager - Python

The package is intended to manage a set of proxies, rotating through at random, to prevent blocking. Proxies are currently dropped from the rotation if they fail twice in a row.

VERY EARLY STAGE

Installation

pip install ProxyManager

Basic usage

PROXY_LIST = [
         '70.45.80.236:4602',
         '70.45.80.236:4603',
         '70.45.80.236:4604'
        ]

URL = 'http://www.myurl.com'

from ProxyManager.core import ProxyManager, Proxy

proxies = [i for i in PROXY_LIST] # OR [Proxy(i) for i in PROXY_LIST]
PM = ProxyManager(proxies)
result = PM.make_request(URL)

# Pass a fail function into make_request method, to check for domain/website specific indicators that the request has failed
# i.e. tell the ProxyManager, that this request has failed
def fail_function(response):
    if '<something_wrong_about>' in response:
        return True
    return False

result = PM.make_request(URL, fail_func=fail_function) # or PM.fail_func = fail_function

Free Proxies

If you don't bring your own proxies, which is HIGHLY recommended, I have written a function that scrapes https://free-proxy-list.net/ to obtain a set of proxies.

from ProxyManager.free_proxies import get_free
FREE_PROXIES = get_free(10)
proxies = [Proxy('{}:{}'.format(i['ip'], i['port'])) for i in FREE_PROXIES]

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for ProxyManager, version 0.0.6
Filename, size File type Python version Upload date Hashes
Filename, size ProxyManager-0.0.6-py3-none-any.whl (9.3 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size ProxyManager-0.0.6.tar.gz (4.7 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page