stickymeta 0.0.5

Handy tools to maintain persistent meta values between requests in Scrapy spiders

take 0.2.0

A DSL for extracting data from a web page.

wayback-machine-scraper 1.0.7

A command-line utility for scraping Wayback Machine snapshots from archive.org.

webcomix 2.1

Webcomic downloader

yy-spider 1.0.0

一个增量式爬虫系统

pyfeeds 0.0.0

scrapinghub-entrypoint-scrapy 0.11.1

Scrapy entrypoint for Scrapinghub job runner

Scrapy 1.5.1

A high-level Web Crawling and Web Scraping framework

Scrapy3 1.0.1

A high-level Web Crawling and Web Scraping framework

scrapy-beautifulsoup 0.0.2

Simple Scrapy middleware to process non-well-formed HTML with BeautifulSoup

scrapy-block-inspector 0.0.2

scrapy-botproxy 1.0.1

BotProxy (IP Rotating HTTP proxy) downloader middlewarefor Scrapy

scrapybox 0.1

A Scrapy GUI

Scrapy-Cookies 0.3

A middleware of cookies persistence for Scrapy

scrapy-crawlera 1.4.0

Crawlera middleware for Scrapy

scrapy-crawl-once 0.1.1

Scrapy middleware which allows to crawl only new content

scrapy-djangoitem 1.1.1

Scrapy extension to write scraped items using Django models

scrapy-do 0.3.0

A daemon for scheduling Scrapy spiders

scrapy-eagle 0.0.37

Run Scrapy Distributed

scrapy-fake-useragent 1.1.0

Use a random User-Agent provided by fake-useragent for every request