A Python SDK for the Scrape.do API
Project description
scrape-do-python
A Python SDK for the Scrape.do web-scraping proxy API, built on httpx and pydantic v2
Quickstart
Install
pip install scrape-do-python
Make a Request
from scrape_do import ScrapeDoClient
# API token is pulled from `SCRAPE_DO_API_KEY`
# (override with `api_token=...`)
with ScrapeDoClient() as client:
response = client.get(
"https://example.com",
super=True,
render=True,
return_json=True,
screenshot=True,
)
response.raise_for_status()
print(response.target_status_code) # 200
print(response.remaining_credits) # 300000.0
response.screenshots[0].to_file("page.png")
Features
| Sync & Async | Sync and Async clients for both API Gateway and Proxy Mode |
| Async API Support | Submit, poll, and fetch jobs against q.scrape.do with the Async API Client |
| Parameter Validation | Every request parameter is validated locally by the RequestParameters pydantic model |
| Typed Responses | ScrapeDoResponse exposes response fields as pydantic models for easy access |
| Browser Automation | Build playWithBrowser sequences from typed Browser Action Models |
| Session Management | Use a custom session_validator function to track proxy rotations when using the session_id parameter
|
| Native Event Hooks | Provide custom request / response / retry / poll hooks that fire at the logical execution boundary |
| Automatic Retries | All clients implement automatic retries on transient gateway/network errors with a customisable Backoff Strategy |
Help
| Documentation | Full API Reference |
| Examples | Curated Usage Examples |
| Roadmap | What's Planned |
| Changelog | What's Already Shipped |
Contributing
Pull Requests, bug reports, and feature requests are all welcome
See
CONTRIBUTINGfor local setup, test commands, and PR conventions
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file scrape_do_python-0.3.0.tar.gz.
File metadata
- Download URL: scrape_do_python-0.3.0.tar.gz
- Upload date:
- Size: 115.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/6.1.0 CPython/3.13.12
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
87075449f2fd3460bf7194d1356fc845cb08c4788e535437065eed2113bff045
|
|
| MD5 |
bbdfb3ea0a5026a7434d8e6e43f8da07
|
|
| BLAKE2b-256 |
5408036ade80d29f0ca414a7469f3c23447ad9c0cfe0118b7523083c7eebf87b
|
Provenance
The following attestation bundles were made for scrape_do_python-0.3.0.tar.gz:
Publisher:
ci.yml on svdC1/scrape-do-python
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
scrape_do_python-0.3.0.tar.gz -
Subject digest:
87075449f2fd3460bf7194d1356fc845cb08c4788e535437065eed2113bff045 - Sigstore transparency entry: 1625714817
- Sigstore integration time:
-
Permalink:
svdC1/scrape-do-python@f7fea3708a237d1a6b681a87f0d8f8e2160166d3 -
Branch / Tag:
refs/tags/v0.3.0 - Owner: https://github.com/svdC1
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
ci.yml@f7fea3708a237d1a6b681a87f0d8f8e2160166d3 -
Trigger Event:
release
-
Statement type:
File details
Details for the file scrape_do_python-0.3.0-py3-none-any.whl.
File metadata
- Download URL: scrape_do_python-0.3.0-py3-none-any.whl
- Upload date:
- Size: 135.4 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/6.1.0 CPython/3.13.12
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
ae1596647a7470ddfa9cf24dea39157c78febf0b205d72a656889d3a46b154bd
|
|
| MD5 |
060cbcc18d503acf60bdfbf512f3d6e7
|
|
| BLAKE2b-256 |
eaad64fd529e7da8f471d416626e24ea7eb266877a589cc2ce8f15cd60e8aa0a
|
Provenance
The following attestation bundles were made for scrape_do_python-0.3.0-py3-none-any.whl:
Publisher:
ci.yml on svdC1/scrape-do-python
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
scrape_do_python-0.3.0-py3-none-any.whl -
Subject digest:
ae1596647a7470ddfa9cf24dea39157c78febf0b205d72a656889d3a46b154bd - Sigstore transparency entry: 1625714896
- Sigstore integration time:
-
Permalink:
svdC1/scrape-do-python@f7fea3708a237d1a6b681a87f0d8f8e2160166d3 -
Branch / Tag:
refs/tags/v0.3.0 - Owner: https://github.com/svdC1
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
ci.yml@f7fea3708a237d1a6b681a87f0d8f8e2160166d3 -
Trigger Event:
release
-
Statement type: