Skip to main content

A Python SDK for the Scrape.do API

Project description

scrape-do-python

PyPI Python License Docs codecov codecov

A Python SDK for the Scrape.do web-scraping proxy API, built on httpx and pydantic v2


Quickstart

Install

pip install scrape-do-python

Make a Request

from scrape_do import ScrapeDoClient

# API token is pulled from `SCRAPE_DO_API_KEY` 

# (override with `api_token=...`)

with ScrapeDoClient() as client:
    response = client.get(
        "https://example.com",
        super=True,
        render=True,
        return_json=True,
        screenshot=True,
    )

response.raise_for_status()
print(response.target_status_code) # 200
print(response.remaining_credits)  # 300000.0
response.screenshots[0].to_file("page.png")

Features

Sync & Async Sync and Async clients for both API Gateway and Proxy Mode
Async API Support Submit, poll, and fetch jobs against q.scrape.do with the Async API Client
Parameter Validation Every request parameter is validated locally by the RequestParameters pydantic model
Typed Responses ScrapeDoResponse exposes response fields as pydantic models for easy access
Browser Automation Build playWithBrowser sequences from typed Browser Action Models
Session Management Use a custom session_validator function to track proxy rotations when using the session_id parameter
Native Event Hooks Provide custom request / response / retry / poll hooks that fire at the logical execution boundary
Automatic Retries All clients implement automatic retries on transient gateway/network errors with a customisable Backoff Strategy

Help

Documentation Full API Reference
Examples Curated Usage Examples
Roadmap What's Planned
Changelog What's Already Shipped

Contributing

Pull Requests, bug reports, and feature requests are all welcome

See CONTRIBUTING for local setup, test commands, and PR conventions

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

scrape_do_python-0.3.0.tar.gz (115.7 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

scrape_do_python-0.3.0-py3-none-any.whl (135.4 kB view details)

Uploaded Python 3

File details

Details for the file scrape_do_python-0.3.0.tar.gz.

File metadata

  • Download URL: scrape_do_python-0.3.0.tar.gz
  • Upload date:
  • Size: 115.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for scrape_do_python-0.3.0.tar.gz
Algorithm Hash digest
SHA256 87075449f2fd3460bf7194d1356fc845cb08c4788e535437065eed2113bff045
MD5 bbdfb3ea0a5026a7434d8e6e43f8da07
BLAKE2b-256 5408036ade80d29f0ca414a7469f3c23447ad9c0cfe0118b7523083c7eebf87b

See more details on using hashes here.

Provenance

The following attestation bundles were made for scrape_do_python-0.3.0.tar.gz:

Publisher: ci.yml on svdC1/scrape-do-python

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file scrape_do_python-0.3.0-py3-none-any.whl.

File metadata

File hashes

Hashes for scrape_do_python-0.3.0-py3-none-any.whl
Algorithm Hash digest
SHA256 ae1596647a7470ddfa9cf24dea39157c78febf0b205d72a656889d3a46b154bd
MD5 060cbcc18d503acf60bdfbf512f3d6e7
BLAKE2b-256 eaad64fd529e7da8f471d416626e24ea7eb266877a589cc2ce8f15cd60e8aa0a

See more details on using hashes here.

Provenance

The following attestation bundles were made for scrape_do_python-0.3.0-py3-none-any.whl:

Publisher: ci.yml on svdC1/scrape-do-python

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page