Skip to main content

# Overview

Author: Niklas Baumstark

dryscrape is a lightweight web scraping library for Python. It uses a headless Webkit instance to evaluate Javascript on the visited pages. This enables painless scraping of plain web pages as well as Javascript-heavy “Web 2.0” applications like Facebook.

It is built on the shoulders of [capybara-webkit](https://github.com/thoughtbot/capybara-webkit)’s [webkit-server](https://github.com/niklasb/webkit-server). A big thanks goes to thoughtbot, inc. for building this excellent piece of software!

# Changelog

  • 1.0: Added Python 3 support, small performance fixes, header names are now properly normalized. Also added the function dryscrape.start_xvfb() to easily start Xvfb.

  • 0.9.1: Changed semantics of the headers function in a backwards-incompatible way: It now returns a list of (key, value) pairs instead of a dictionary.

# Supported Platforms

The library has been confirmed to work on the following platforms:

  • Mac OS X 10.9 Mavericks and 10.10 Yosemite

  • Ubuntu Linux

  • Arch Linux

Other unixoid systems should work just fine.

Windows is not officially supported, although dryscrape should work with [cygwin](https://www.cygwin.com/).

# Installation, Usage, API Docs

Documentation can be found at [dryscrape’s ReadTheDocs page](http://readthedocs.org/docs/dryscrape/).

Quick installation instruction:

# pip install dryscrape

# Contact, Bugs, Contributions

If you have any problems with this software, don’t hesitate to open an issue on [Github](https://github.com/niklasb/dryscrape) or open a pull request or write a mail to niklas baumstark at Gmail.

Metadata

Release files for dryscrape 1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for dryscrape 1.0
File Size Uploaded Provenance
dryscrape-1.0.tar.gz 5.5 kB Without provenance Details

Release files / dryscrape-1.0.tar.gz

Download URL dryscrape-1.0.tar.gz
Size 5.5 kB
Tags Source
SHA-256 checksum
How to use checksums
a99858786434947266cb81d5634cb1722de48aaf6b9cdffda15b7cd4a8e07340
BLAKE2b-256 checksum
How to use checksums
b575c45f796ec5bc7f98c38b9ae425390ef5f4a76153c8b5af946adb97e7e622
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No

Security for dryscrape 1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

1.0 This release

1 release file No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page