Skip to main content
Avatar for scrapy from gravatar.com
Username    scrapy

28 projects

itemadapter

Last released on

Common interface for data container classes

itemloaders

Last released on

Base library for scrapy's ItemLoader

w3lib

Last released on

Library of web-related functions

parsel

Last released on

Parsel is a library to extract data from HTML and XML using XPath and CSS selectors

scrapy-poet

Last released on

Page Object pattern for Scrapy

web-poet

Last released on

Scrapinghub's Page Object pattern for web scraping

Scrapy

Last released on

A high-level Web Crawling and Web Scraping framework

andi

Last released on

Library for annotation-based dependency injection

splash

Last released on

A javascript rendered with a HTTP API

Protego

Last released on

Pure-Python robots.txt parser with support for modern conventions

scrapely

Last released on

A pure-python HTML screen-scraping library

scrapy-po

Last released on

Page Object pattern for Scrapy

cssselect

Last released on

cssselect parses CSS3 Selectors and translates them to XPath 1.0

scrapyd

Last released on

A service for running Scrapy spiders, with an HTTP API

queuelib

Last released on

Collection of persistent (disk-based) queues

webstruct

Last released on

A library for creating statistical NER systems that work on HTML data

scrapyd-client

Last released on

A client for scrapyd

PyPyDispatcher

Last released on

Multi-producer-multi-consumer signal dispatching mechanism

scrapy-deltafetch

Last released on

Scrapy middleware to ignore previously crawled pages

adblockparser

Last released on

Parser for Adblock Plus rules

loginform

Last released on

Fill HTML login forms automatically

scrapy-splitvariants

Last released on

Scrapy spider middleware to split an item into multiple items on a multi-valued key

scrapy-hcf

Last released on

Scrapy spider middleware to use Scrapinghub's Hub Crawl Frontier as a backend for URLs

scrapy-querycleaner

Last released on

Scrapy spider middleware to clean up query parameters in request URLs

scrapy-magicfields

Last released on

Scrapy middleware to add extra "magic" fields to items

scrapy-djangoitem

Last released on

Scrapy extension to write scraped items using Django models

scrapyjs

Last released on

JavaScript support for Scrapy using Splash

scrapy-jsonrpc

Last released on

Scrapy extenstion to control spiders using JSON-RPC

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page