This is a pre-production deployment of Warehouse, however changes made here WILL affect the production instance of PyPI.
scrapy

scrapy

Projects

adblockparser

Last released on Oct 17, 2016

Parser for Adblock Plus rules

cssselect

Last released on Oct 21, 2016

cssselect parses CSS3 Selectors and translates them to XPath 1.0

loginform

Last released on Aug 16, 2016

Fill HTML login forms automatically

parsel

Last released on Nov 22, 2016

Parsel is a library to extract data from HTML and XML using XPath and CSS selectors

queuelib

Last released on Sep 9, 2015

Collection of persistent (disk-based) queues

scrapely

Last released on Jan 26, 2015

A pure-python HTML screen-scraping library

Scrapy

Last released on Oct 21, 2016

A high-level Web Crawling and Web Scraping framework

scrapyd

Last released on Nov 3, 2016

A service for running Scrapy spiders, with an HTTP API

scrapyd-client

Last released on Apr 9, 2015

A client for scrapyd

scrapy-deltafetch

Last released on Jun 29, 2016

Scrapy middleware to ignore previously crawled pages

scrapy-djangoitem

Last released on May 4, 2016

Scrapy extension to write scraped items using Django models

scrapy-hcf

Last released on Jul 18, 2016

Scrapy spider middleware to use Scrapinghub's Hub Crawl Frontier as a backend for URLs

scrapyjs

Last released on Mar 25, 2016

JavaScript support for Scrapy using Splash

scrapy-jsonrpc

Last released on Apr 13, 2015

Scrapy extenstion to control spiders using JSON-RPC

scrapy-magicfields

Last released on Jun 30, 2016

Scrapy middleware to add extra "magic" fields to items

scrapy-querycleaner

Last released on Jun 30, 2016

Scrapy spider middleware to clean up query parameters in request URLs

scrapy-splitvariants

Last released on Jul 18, 2016

Scrapy spider middleware to split an item into multiple items on a multi-valued key

splash

Last released on Nov 30, 2016

A javascript rendered with a HTTP API

w3lib

Last released on Nov 10, 2016

Library of web-related functions

webstruct

Last released on Nov 28, 2016

A library for creating statistical NER systems that work on HTML data

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS HPE HPE Development Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting