with the selected classifier
Handy tools to maintain persistent meta values between requests in Scrapy spiders
A DSL for extracting data from a web page.
A command-line utility for scraping Wayback Machine snapshots from archive.org.
Scrapy entrypoint for Scrapinghub job runner
A high-level Web Crawling and Web Scraping framework
Simple Scrapy middleware to process non-well-formed HTML with BeautifulSoup
BotProxy (IP Rotating HTTP proxy) downloader middlewarefor Scrapy
A Scrapy GUI
A middleware of cookies persistence for Scrapy
Crawlera middleware for Scrapy
Scrapy middleware which allows to crawl only new content
Scrapy extension to write scraped items using Django models
A daemon for scheduling Scrapy spiders
Run Scrapy Distributed
Use a random User-Agent provided by fake-useragent for every request