A webarchive extension for Scrapy

Navigation

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for wvengen from gravatar.com wvengen

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • Tags Scrapy , Webarchive , WARC , WACZ
  • Requires: Python <3.13, >=3.7
  • Provides-Extra: all , aws , gcs
Classifiers
Report project as malware

Project description

Scrapy Webarchive

Docs

Scrapy Webarchive is a plugin for Scrapy that allows users to capture and export web archives in the WARC and WACZ formats during crawling.

Features

  • Save web crawls in WACZ format (multiple storages supported; local and cloud).
  • Crawl against WACZ format archives.
  • Integrate seamlessly with Scrapy’s spider request and response cycle.

Compatibility

  • Python 3.7, 3.8, 3.9, 3.10, 3.11 and 3.12

Documentation

Documentation is available online at developers.thequestionmark.org/scrapy-webarchive/

Project details

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for wvengen from gravatar.com wvengen

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • Tags Scrapy , Webarchive , WARC , WACZ
  • Requires: Python <3.13, >=3.7
  • Provides-Extra: all , aws , gcs
Classifiers

Release history Release notifications | RSS feed

0.5.2

0.5.1

This version

0.5.0

0.4.1

0.4.0

0.3.0

0.2.0

0.1.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

scrapy_webarchive-0.5.0.tar.gz (27.6 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

scrapy_webarchive-0.5.0-py3-none-any.whl (29.7 kB view details)

Uploaded Python 3

File details

Details for the file scrapy_webarchive-0.5.0.tar.gz.

File metadata

  • Download URL: scrapy_webarchive-0.5.0.tar.gz
  • Upload date:
  • Size: 27.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for scrapy_webarchive-0.5.0.tar.gz
Algorithm Hash digest
SHA256 66e1e3bd71fd6580332e045647c4db4b2739ac156bc91683240f6d713aa0071f
MD5 b28d3b76026a7098a8538231a165e087
BLAKE2b-256 96acf232a1dc0f2323a818b59a9d22b77dcbc6a11b4d89f845e09737cc1e7485

See more details on using hashes here.

Provenance

The following attestation bundles were made for scrapy_webarchive-0.5.0.tar.gz:

Publisher: release.yml on q-m/scrapy-webarchive

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file scrapy_webarchive-0.5.0-py3-none-any.whl.

File metadata

File hashes

Hashes for scrapy_webarchive-0.5.0-py3-none-any.whl
Algorithm Hash digest
SHA256 93c5df32d3f2771dbf9706b938a5265c1a10c5acbb384ddf9ed5349fe7264fa7
MD5 59949ca77453db14fbd0a9c8c4e5954f
BLAKE2b-256 719780a16b6df41ac4cc457eb97d015229e181d4a117f62b9eec4e7e422adabc

See more details on using hashes here.

Provenance

The following attestation bundles were made for scrapy_webarchive-0.5.0-py3-none-any.whl:

Publisher: release.yml on q-m/scrapy-webarchive

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page