A description of your project
Project description
title: Title
summary: "summary"
Proxy Manager - Python
The package is intended to manage a set of proxies, rotating through at random, to prevent blocking. Proxies are currently dropped from the rotation if they fail twice in a row.
VERY EARLY STAGE
Installation
pip install ProxyManager
Basic usage
PROXY_LIST = [
'70.45.80.236:4602',
'70.45.80.236:4603',
'70.45.80.236:4604'
]
URL = 'http://www.myurl.com'
from ProxyManager.core import ProxyManager, Proxy
proxies = [i for i in PROXY_LIST] # OR [Proxy(i) for i in PROXY_LIST]
PM = ProxyManager(proxies)
result = PM.make_request(URL)
# Pass a fail function into make_request method, to check for domain/website specific indicators that the request has failed
# i.e. tell the ProxyManager, that this request has failed
def fail_function(response):
if '<something_wrong_about>' in response:
return True
return False
result = PM.make_request(URL, fail_func=fail_function) # or PM.fail_func = fail_function
Free Proxies
If you don't bring your own proxies, which is HIGHLY recommended, I have written a function that scrapes https://free-proxy-list.net/ to obtain a set of proxies.
from ProxyManager.free_proxies import get_free
FREE_PROXIES = get_free(10)
proxies = [Proxy('{}:{}'.format(i['ip'], i['port'])) for i in FREE_PROXIES]
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
ProxyManager-0.0.6.tar.gz
(4.7 kB
view hashes)
Built Distribution
Close
Hashes for ProxyManager-0.0.6-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c7a92926cb10645187a5783ac6528b28f03334ba34879d76c32b3b776599172e |
|
MD5 | 39324662d1daf108179c35a0cb650578 |
|
BLAKE2b-256 | 002105d9074db71227f8ce5c1bf6d226974140b01552a5929026de32d907f93c |