PyPI Version Supported Python Versions Build Status Wheel Status Coverage report Conda Version

Overview

Scrapy is a fast high-level web crawling and web scraping framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.

For more information including a list of features check the Scrapy homepage at: https://scrapy.org

Requirements

  • Python 2.7 or Python 3.4+

  • Works on Linux, Windows, Mac OSX, BSD

Install

The quick way:

pip install scrapy

For more details see the install section in the documentation: https://doc.scrapy.org/en/latest/intro/install.html

Documentation

Documentation is available online at https://doc.scrapy.org/ and in the docs directory.

Releases

You can find release notes at https://doc.scrapy.org/en/latest/news.html

Community (blog, twitter, mail list, IRC)

See https://scrapy.org/community/

Contributing

See https://doc.scrapy.org/en/master/contributing.html

Code of Conduct

Please note that this project is released with a Contributor Code of Conduct (see https://github.com/scrapy/scrapy/blob/master/CODE_OF_CONDUCT.md).

By participating in this project you agree to abide by its terms. Please report unacceptable behavior to opensource@scrapinghub.com.

Companies using Scrapy

See https://scrapy.org/companies/

Commercial Support

See https://scrapy.org/support/

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (2.18.0)

4 maintainers

PyPI data
Data sourced directly from PyPI's database.
Avatar for dangra from gravatar.com dangra Avatar for kmike from gravatar.com kmike Avatar for pablohoffman from gravatar.com pablohoffman
+1 more Avatar for scrapy from gravatar.com scrapy

Credits

Author: Scrapy developers

Maintainer: Pablo Hoffman

License

BSD License (BSD)

Requires

Python >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

Scrapy-1.6.0.tar.gz (926.6 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

Scrapy-1.6.0-py2.py3-none-any.whl (231.8 kB view details)

Uploaded Python 2Python 3

File details

Details for the file Scrapy-1.6.0.tar.gz.

File metadata

  • Download URL: Scrapy-1.6.0.tar.gz
  • Upload date:
  • Size: 926.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.12.1 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.7.1 requests-toolbelt/0.9.1 tqdm/4.30.0 CPython/2.7.14

File hashes

Hashes for Scrapy-1.6.0.tar.gz
Algorithm Hash digest
SHA256 558dfd10ac53cb324ecd7eefd3eac412161c7507c082b01b0bcd2c6e2e9f0766
MD5 43143373ef278a1e6ad941b1c84c969b
BLAKE2b-256 698fd55f61bb3ba687fa440186195db8287a87f5e79c15710d51b7ed1a342dd8

See more details on using hashes here.

File details

Details for the file Scrapy-1.6.0-py2.py3-none-any.whl.

File metadata

  • Download URL: Scrapy-1.6.0-py2.py3-none-any.whl
  • Upload date:
  • Size: 231.8 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.12.1 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.7.1 requests-toolbelt/0.9.1 tqdm/4.30.0 CPython/2.7.14

File hashes

Hashes for Scrapy-1.6.0-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 4ec03552c9aed1e82a376488150e904e0213bdee3ca140225105e9b03d0de204
MD5 dc929440dab468f44e64386dd10a756c
BLAKE2b-256 3e45414e87ac8209d537c91575538c5307c20217a6943f555e0ee39f6db4bb0f

See more details on using hashes here.

Release history Release notifications | RSS feed

2.18.0

2 files

2.17.0

2 files

2.16.0

2 files

2.15.2

2 files

2.15.1

2 files

2.15.0

2 files

2.14.2

2 files

2.14.1

2 files

2.14.0

2 files

2.13.4

2 files

2.13.3

2 files

2.13.2

2 files

2.13.1

2 files

2.13.0

2 files

2.12.0

2 files

2.11.2

2 files

2.11.1

2 files

2.11.0

2 files

2.10.1

2 files

2.10.0

2 files

2.9.0

2 files

2.8.0

2 files

2.7.1

2 files

2.7.0

2 files

2.6.3

2 files

2.6.2

2 files

2.6.1

2 files

2.6.0

2 files

2.5.1

2 files

2.5.0

2 files

2.4.1

2 files

2.4.0

2 files

2.3.0

2 files

2.2.1

2 files

2.2.0

2 files

2.1.0

2 files

2.0.1

2 files

2.0.0

2 files

1.8.4

2 files

1.8.3

2 files

1.8.2

2 files

1.8.1

2 files

1.8.0

2 files

1.7.4

2 files

1.7.3

2 files

1.7.2

2 files

1.7.1

2 files

1.7.0

2 files

This release

1.6.0 This release

2 files

1.5.2

2 files

1.5.1

2 files

1.5.0

2 files

1.4.0

2 files

1.3.3

2 files

1.3.2

2 files

1.3.1

2 files

1.3.0

2 files

1.2.3

2 files

1.2.2

2 files

1.2.1

2 files

1.2.0

2 files

1.1.4

2 files

1.1.3

2 files

1.1.2

2 files

1.1.1

2 files

1.1.0

2 files

Pre-release

1.1.0rc4

1 file

Pre-release

1.1.0rc3

2 files

Pre-release

1.1.0rc2

1 file

Pre-release

1.1.0rc1

2 files

1.0.7

2 files

1.0.6

2 files

1.0.5

2 files

1.0.4

2 files

1.0.3

2 files

1.0.2

2 files

1.0.1

2 files

1.0.0

2 files

Pre-release

1.0.0rc3

2 files

Pre-release

1.0.0rc2

2 files

Pre-release

1.0.0rc1

2 files

0.24.6

2 files

0.24.5

2 files

0.24.4

2 files

0.24.3

2 files

0.24.2

2 files

0.24.1

2 files

0.24.0

2 files

0.22.2

1 file

0.22.1

1 file

0.22.0

1 file

0.20.2

1 file

0.20.1

1 file

0.20.0

1 file

0.18.4

1 file

0.18.3

1 file

0.18.2

1 file

0.18.1

1 file

0.18.0

1 file

0.16.5

1 file

0.16.4

1 file

0.16.3

1 file

0.16.2

1 file

0.16.1

1 file

0.16.0

1 file

0.14.4

1 file

0.14.3

2 files

0.14.2

2 files

0.14.1

2 files

0.12.0.2550

1 file

0.10.4.2364

1 file

0.9

2 files

0.8

2 files

0.7

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page