proxy-middleware
Scrapy middlewares that reads proxy config from settings.
Install
pip install proxy-middleware
ProxyFromSettingsMiddleware
A middleware that sets proxy from settings file.
Usage: add it to DOWNLOADER_MIDDLEWARES in scrapy settings:
DOWNLOADER_MIDDLEWARES = {
'proxy_middleware.ProxyFromSettingsMiddleware': 10,
...
Pass proxy config via HTTP_PROXY and HTTPS_PROXY settings variables. HTTPPROXY_AUTH_ENCODING is also respected:
scrapy crawl my-spider -s HTTP_PROXY=http://localhost:8118
ProxyOnlyTorMiddleware
A middleware that proxies only requests to TOR sites (domain ends with “.onion”).
Usage: add it to DOWNLOADER_MIDDLEWARES in scrapy settings:
DOWNLOADER_MIDDLEWARES = {
'proxy_middleware.ProxyOnlyTorMiddleware': 10,
...
Settings: HTTP_PROXY for an http proxy and HTTPS_PROXY for an https proxy.
License
License is MIT.
History
0.2.0 (2017-05-29)
ProxyOnlyTorMiddleware added.
0.1.1 (2017-04-07)
Initial release
Release files for proxy-middleware 0.2.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| proxy-middleware-0.2.0.tar.gz | 2.4 kB | Details |
Release files / proxy-middleware-0.2.0.tar.gz
| Download URL | proxy-middleware-0.2.0.tar.gz |
|---|---|
| Size | 2.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
2cf6a0531af0cc728208fc247a2e0e33e9ac9ff771d4d4407e5622d8ebe676da
|
|
BLAKE2b-256 checksum How to use checksums |
ad910004924204956f08365e4c55dc07ed3f8ced2d23baeede0cceece2177294
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
