Crawlera middleware for Scrapy
Project description
scrapy-crawlera provides easy use of Crawlera with Scrapy.
Installation
You can install scrapy-crawlera using pip:
pip install scrapy-crawlera
You can then enable the middleware in your settings.py:
DOWNLOADER_MIDDLEWARES = { ... 'crawlera.CrawleraMiddleware': 600 }
There are two ways to specify credentials.
Through settings.py:
CRAWLERA_ENABLED = True CRAWLERA_USER = 'username' CRAWLERA_PASS = 'password'
Through spider attributes:
class MySpider: crawlera_enabled = True crawlera_user = 'username' crawlera_pass = 'password'
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
scrapy-crawlera-1.0.2.tar.gz
(3.1 kB
view hashes)