let me crawl
Project description
Curate proxies from various sources in terms of latency. Supported on python 2.7.* and >3.4.
>> from letmecrawl import letmecrawl
>> for proxy in letmecrawl():
>> print(proxy)
88.99.11.10:6666
Installation
pip install letmecrawl
Docker
Use docker image as a http service:
docker pull montenegrodr/letmecrawl:latest
Running:
docker run -p 8000:8000 montenegrodr/letmecrawl:latest
Request proxy from top of the queue:
$ curl localhost:8000
{"proxy":"138.68.178.106:80"}
$ curl localhost:8000
{"proxy":"184.59.102.95:80"}
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
letmecrawl-0.13.tar.gz
(4.5 kB
view hashes)