Profile of scrapy

Scrapy

Last released Nov 18, 2024

A high-level Web Crawling and Web Scraping framework

web-poet

Last released Oct 11, 2024

Zyte's Page Object pattern for web scraping

scrapyd-client

Last released Oct 11, 2024

A client for Scrapyd

scrapy-poet

Last released Oct 10, 2024

Page Object pattern for Scrapy

scrapyd

Last released Oct 2, 2024

A service for running Scrapy spiders, with an HTTP API

itemloaders

Last released Sep 30, 2024

Base library for scrapy's ItemLoader

scrapy-zyte-smartproxy

Last released Aug 5, 2024

Scrapy middleware for Zyte Smart Proxy Manager

form2request

Last released Jul 15, 2024

Build HTTP requests out of HTML forms

w3lib

Last released Jun 12, 2024

Library of web-related functions

itemadapter

Last released May 7, 2024

Common interface for data container classes

queuelib

Last released May 4, 2024

Collection of persistent (disk-based) and non-persistent (memory-based) queues

parsel

Last released Apr 8, 2024

Parsel is a library to extract data from HTML and XML using XPath and CSS selectors

Protego

Last released Apr 5, 2024

Pure-Python robots.txt parser with support for modern conventions

xtractmime

Last released Jan 16, 2024

Implementation of the MIME Sniffing standard (https://mimesniff.spec.whatwg.org/)

andi

Last released Dec 26, 2023

Library for annotation-based dependency injection

scrapy-splash

Last released Feb 3, 2023

JavaScript support for Scrapy using Splash

cssselect

Last released Oct 27, 2022

cssselect parses CSS3 Selectors and translates them to XPath 1.0

scrapy-deltafetch

Last released Sep 20, 2021

Scrapy middleware to ignore previously crawled pages

splash

Last released Jun 16, 2020

A javascript rendered with a HTTP API

scrapely

Last released Nov 28, 2019

A pure-python HTML screen-scraping library

scrapy-po

Last released Aug 28, 2019

Page Object pattern for Scrapy

webstruct

Last released Dec 29, 2017

A library for creating statistical NER systems that work on HTML data

PyPyDispatcher

Last released Jul 3, 2017

Multi-producer-multi-consumer signal dispatching mechanism

adblockparser

Last released Oct 17, 2016

Parser for Adblock Plus rules

loginform

Last released Aug 16, 2016

Fill HTML login forms automatically

scrapy-splitvariants

Last released Jul 18, 2016

Scrapy spider middleware to split an item into multiple items on a multi-valued key

scrapy-hcf