Proxy Port Scrapy middleware
Project description
Proxy Port integration for Scrapy.
Prerequisites
To use this package you will need a free API key. Get your API key here. Detailed instructions here.
Installation
$ pip install scrapyproxyport
Getting Started
You need to assign an API key. This can be done either through an environment variable
$ export PROXY_PORT_API_KEY=<API_KEY>
or inside settings.py.
# inside <your_project>/settings.py
PROXY_PORT_API_KEY = '<API_KEY>'
DOWNLOADER_MIDDLEWARES = {
# Add middleware with order number right before CookiesMiddleware
'scrapyproxyport.middlewares.ProxyMiddleware': 898,
'scrapy.downloadermiddlewares.cookies.CookiesMiddleware': 899,
}
DOWNLOAD_TIMEOUT = 10
RETRY_TIMES = 20
DUPEFILTER_CLASS = 'scrapy.dupefilters.BaseDupeFilter'
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
scrapyproxyport-1.1.0.tar.gz
(4.6 kB
view hashes)
Built Distribution
Close
Hashes for scrapyproxyport-1.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4ef5e29c18154211206d01d659cef6bb22740af96ae9bc2e97925f0c597f45c6 |
|
MD5 | 8a7b7aebda7c5ce106e10e4113fbd5ef |
|
BLAKE2b-256 | 4d4d8691771da3d61a8db6f7ce9ac877a5045ad37517a9c3a620bb0c38a4559b |