Auto Generated by os-scrapy-cookiecutter
Project description
os-scrapy-httpproxy-middleware
This project provide middleware to enhance Scrapy built-in HttpProxyMiddleware. You can set proxy without sheme as the following format:
[user:password@]proxy:port
Install
pip install os-scrapy-httpproxy-middleware
You can run example spider directly in the project root path
scrapy crawl example
Usage
Settings
-
enable middleware, it is better disable Scrapy built-in HttpProxyMiddleware
DOWNLOADER_MIDDLEWARES = { "scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware": None, "os_scrapy_httpproxy_middleware.middlewares.HttpProxyMiddleware": 750, }
Unit Tests
sh scripts/test.sh
License
MIT licensed.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Close
Hashes for os_scrapy_httpproxy_middleware-0.0.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | e3daa3b8bc342d301e1400b6a1ae452348304924be3cbab62f3aa34c54fc1030 |
|
MD5 | 023046357fd4b89f549cace6f0e76bf1 |
|
BLAKE2b-256 | 4927e4bd305338d8c1809d2ed3b46ad06bc9d7db8bb4980cab2a2660cc47b75a |