Skip to main content

A Python SDK for the Scrape.do API

Project description

scrape-do-python

PyPI Python License Docs codecov codecov

A Python SDK for the Scrape.do web-scraping proxy API, built on httpx and pydantic v2


Quickstart

Install

pip install scrape-do-python

Make a Request

from scrape_do import ScrapeDoClient

# API token is pulled from `SCRAPE_DO_API_KEY` 

# (override with `api_token=...`)

with ScrapeDoClient() as client:
    response = client.get(
        "https://example.com",
        super=True,
        render=True,
        return_json=True,
        screenshot=True,
    )

response.raise_for_status()
print(response.target_status_code) # 200
print(response.remaining_credits)  # 300000.0
response.screenshots[0].to_file("page.png")

Features

Sync & Async Sync and Async clients for both API Gateway and Proxy Mode
Async API Support Submit, poll, and fetch jobs against q.scrape.do with the Async API Client
Parameter Validation Every request parameter is validated locally by the RequestParameters pydantic model
Typed Responses ScrapeDoResponse exposes response fields as pydantic models for easy access
Browser Automation Build playWithBrowser sequences from typed Browser Action Models
Session Management Use a custom session_validator function to track proxy rotations when using the session_id parameter
Native Event Hooks Provide custom request / response / retry / poll hooks that fire at the logical execution boundary
Automatic Retries All clients implement automatic retries on transient gateway/network errors with a customisable Backoff Strategy

Help

Documentation Full API Reference
Examples Curated Usage Examples
Roadmap What's Planned
Changelog What's Already Shipped

Contributing

Pull Requests, bug reports, and feature requests are all welcome

See CONTRIBUTING for local setup, test commands, and PR conventions

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

scrape_do_python-0.3.1.tar.gz (123.5 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

scrape_do_python-0.3.1-py3-none-any.whl (145.2 kB view details)

Uploaded Python 3

File details

Details for the file scrape_do_python-0.3.1.tar.gz.

File metadata

  • Download URL: scrape_do_python-0.3.1.tar.gz
  • Upload date:
  • Size: 123.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for scrape_do_python-0.3.1.tar.gz
Algorithm Hash digest
SHA256 fbeb2a6b109815f803b4188846caedeea3ba1f286bd016f48a7eb5a041e67770
MD5 ccdee675d27f78e1081ae63eb12b595d
BLAKE2b-256 f3800f54271d34029cdc7dadcc485ec177f03e595d3f0ca41a7cd8ce82ddbf86

See more details on using hashes here.

Provenance

The following attestation bundles were made for scrape_do_python-0.3.1.tar.gz:

Publisher: ci.yml on svdC1/scrape-do-python

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file scrape_do_python-0.3.1-py3-none-any.whl.

File metadata

File hashes

Hashes for scrape_do_python-0.3.1-py3-none-any.whl
Algorithm Hash digest
SHA256 f252564baa011a9bbdbd353edb57c0bee248e0ef23bbc35ac81e25f582c30220
MD5 facb4567526a30c9e2970847ebbd2c2e
BLAKE2b-256 3881313e9fa814df2c6845e71edc34b8b951fa2e205ed80f01d5ad4052f13afa

See more details on using hashes here.

Provenance

The following attestation bundles were made for scrape_do_python-0.3.1-py3-none-any.whl:

Publisher: ci.yml on svdC1/scrape-do-python

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page