WebCrawler API Python SDK
A Python SDK for interacting with the WebCrawlerAPI.
Get your API key at dash.webcrawlerapi.com/access.
Installation
pip install webcrawlerapi
Crawling
from webcrawlerapi import WebCrawlerAPI
client = WebCrawlerAPI(api_key="your_api_key")
job = client.crawl(
url="https://example.com",
output_formats=["markdown"],
items_limit=10,
)
for item in job.job_items:
if item.status == "done":
print(item.original_url)
print(item.content)
Scraping
from webcrawlerapi import WebCrawlerAPI
client = WebCrawlerAPI(api_key="your_api_key")
result = client.scrape(
url="https://example.com",
output_formats=["markdown"],
)
if result.success:
print(result.markdown)
else:
print(f"Error: {result.error_code} — {result.error_message}")
Documentation
- Crawling — parameters, async, job/item responses, cancellation
- Scraping — parameters, async, response fields
- Advanced — output formats, structured outputs, webhooks, S3 actions
Requirements
- Python 3.6+
- requests >= 2.25.0
License
MIT License
1 maintainer
Metadata
1 maintainer
Release files for webcrawlerapi 2.1.5
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| webcrawlerapi-2.1.5.tar.gz | 14.6 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| webcrawlerapi-2.1.5-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 30.1 kB
Release files / webcrawlerapi-2.1.5.tar.gz
| Download URL | webcrawlerapi-2.1.5.tar.gz |
|---|---|
| Size | 14.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
c92d94b9ba7a7dce4ea71c462aaf7d3184257f458b5946b832d9dc64e30c1d60
|
|
BLAKE2b-256 checksum How to use checksums |
3332efb241dc892c20776759752215285c1d7dd102c463004f01dc2536808697
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.14.7
|
Release files / webcrawlerapi-2.1.5-py3-none-any.whl
| Download URL | webcrawlerapi-2.1.5-py3-none-any.whl |
|---|---|
| Size | 15.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
21966bc725f6f06074394afae03fde1f30cde3c5f78b82130f7da99e8cb67291
|
|
BLAKE2b-256 checksum How to use checksums |
e66ea686122a996645f8d9dd6a242a53d6c1218915837f95ef9efc836ded8e69
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.14.7
|
