Skip to main content

A high-level Web Crawling and Web Scraping framework

Project description

PyPI Version Supported Python Versions Build Status Wheel Status Coverage report Conda Version

Abstract

This is a fork for Scrapy. For apply best in python3

Overview

Scrapy is a fast high-level web crawling and web scraping framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.

For more information including a list of features check the Scrapy homepage at: https://scrapy.org

Requirements

  • Python 3.4+
  • Works on Linux, Windows, Mac OSX, BSD

Install

The quick way:

pip install scrapy

For more details see the install section in the documentation: https://doc.scrapy.org/en/latest/intro/install.html

Documentation

Documentation is available online at https://doc.scrapy.org/ and in the docs directory.

Releases

You can find release notes at https://doc.scrapy.org/en/latest/news.html

Community (blog, twitter, mail list, IRC)

See https://scrapy.org/community/

Contributing

See https://doc.scrapy.org/en/master/contributing.html

Code of Conduct

Please note that this project is released with a Contributor Code of Conduct (see https://github.com/scrapy/scrapy/blob/master/CODE_OF_CONDUCT.md).

By participating in this project you agree to abide by its terms. Please report unacceptable behavior to opensource@scrapinghub.com.

Companies using Scrapy

See https://scrapy.org/companies/

Commercial Support

See https://scrapy.org/support/

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for Scrapy3, version 1.0.1
Filename, size File type Python version Upload date Hashes
Filename, size Scrapy3-1.0.1-py2.py3-none-any.whl (230.7 kB) File type Wheel Python version py2.py3 Upload date Hashes View
Filename, size Scrapy3-1.0.1.tar.gz (912.7 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page