Scrapy downloader middleware that rotates proxies.
Project description
This is a Scrapy downloader middleware that sets proxy server for requests. Currently proxy servers are rotated randomly. In the future more rotation strategies will be supported.
Configuration
Scrapy settings file:
DOWNLOADER_MIDDLEWARES = { 'scrapy_proxy_rotator.ProxyMiddleware': 1, } PROXY_ROTATOR = { 'username': 'user1', 'password': 'pass1', 'proxies_file': 'proxies.txt', }
In proxies file proxy server must be formatted like this http://host:port.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Close
Hashes for scrapy-proxy-rotator-0.1.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | e5af52cf003bcda44a6c2c488c09e7c95eeb88483b3c94ad971d972bc19bc6c7 |
|
MD5 | bf87103cda738f208dbba20f1820c2c7 |
|
BLAKE2b-256 | ed857527b898ea09332f62b40c351e4e468126767c0c743db6d043a1bd4c9239 |