Free HTTP proxy
Project description
Key Features
Provides proxy agent (server) to scrap free HTTP proxies and verify the usability of each of them.
Provides proxy pool (client) to collect the proxies from agent and maintains them locally.
Installation
Use pip:
$ pip install freehp
Use source code:
$ python setup.py install
Getting Started
Proxy Agent
Run proxy agent:
$ freehp run
The proxy agent by default runs on port 8081. Then we can visit http://localhost:8081/ and see a list of latest available proxies.
Proxy Pool
An usage example of SimpleProxyPool:
from freehp import SimpleProxyPool
from freehp.errors import NoProxyAvailable
if __name__ == '__main__':
pool = SimpleProxyPool("http://localhost:8081")
try:
proxy = pool.get_proxy()
print("The proxy is: {}".format(proxy))
except NoProxyAvailable:
print("No proxy available now")
SimpleProxyPool randomly selects the proxy in the list each time.
An usage example of ProxyPool:
from random import randint
from freehp import ProxyPool
from freehp.errors import NoProxyAvailable
if __name__ == '__main__':
pool = ProxyPool("http://localhost:8081")
try:
proxy = pool.get_proxy()
print("The proxy is: {}".format(proxy))
except NoProxyAvailable:
print("No proxy available now")
else:
# connect the website using the proxy ...
# ok is `True` if the connection succeeded, otherwise `False`
ok = bool(randint(0, 1))
print("Connection {}".format("succeeded" if ok else "failed"))
# feed back the availability of the proxy
pool.feed_back(proxy, ok)
ProxyPool prefers to select the proxy with high connection success rate.
Requirements
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
freehp-0.3.0.tar.gz
(19.0 kB
view hashes)