This is a Scrapy downloader middleware that sets proxy server for requests. Currently proxy servers are rotated randomly. In the future more rotation strategies will be supported.
Configuration
Scrapy settings file:
DOWNLOADER_MIDDLEWARES = {
'scrapy_proxy_rotator.ProxyMiddleware': 1,
}
PROXY_ROTATOR = {
'username': 'user1',
'password': 'pass1',
'proxies_file': 'proxies.txt',
}
In proxies file proxy server must be formatted like this http://host:port.
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
File details
Details for the file scrapy-proxy-rotator-0.1.0.tar.gz.
File metadata
- Download URL: scrapy-proxy-rotator-0.1.0.tar.gz
- Upload date:
- Size: 2.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
e5af52cf003bcda44a6c2c488c09e7c95eeb88483b3c94ad971d972bc19bc6c7
|
|
| MD5 |
bf87103cda738f208dbba20f1820c2c7
|
|
| BLAKE2b-256 |
ed857527b898ea09332f62b40c351e4e468126767c0c743db6d043a1bd4c9239
|
