WebCrawler API Python SDK
A Python SDK for interacting with the WebCrawlerAPI.
Get your API key at dash.webcrawlerapi.com/access.
Installation
pip install webcrawlerapi
Crawling
from webcrawlerapi import WebCrawlerAPI
client = WebCrawlerAPI(api_key="your_api_key")
job = client.crawl(
url="https://example.com",
output_formats=["markdown"],
items_limit=10,
)
for item in job.job_items:
if item.status == "done":
print(item.original_url)
print(item.content)
Scraping
from webcrawlerapi import WebCrawlerAPI
client = WebCrawlerAPI(api_key="your_api_key")
result = client.scrape(
url="https://example.com",
output_formats=["markdown"],
)
if result.success:
print(result.markdown)
else:
print(f"Error: {result.error_code} — {result.error_message}")
Documentation
- Crawling — parameters, async, job/item responses, cancellation
- Scraping — parameters, async, response fields
- Advanced — output formats, structured outputs, webhooks, S3 actions
Requirements
- Python 3.6+
- requests >= 2.25.0
License
MIT License
1 maintainer
Metadata
1 maintainer
Release files for webcrawlerapi 2.1.3
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| webcrawlerapi-2.1.3.tar.gz | 14.4 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| webcrawlerapi-2.1.3-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 29.9 kB
Release files / webcrawlerapi-2.1.3.tar.gz
| Download URL | webcrawlerapi-2.1.3.tar.gz |
|---|---|
| Size | 14.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
82c2a304f205f1600a48563cf5a7ef0b1a8524026ff26f8741d367668d7b47ff
|
|
BLAKE2b-256 checksum How to use checksums |
6fafb3f64acc029b9f6d40f33e841103a6136835164d0f08d525fedbf62f6d34
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.14.6
|
Release files / webcrawlerapi-2.1.3-py3-none-any.whl
| Download URL | webcrawlerapi-2.1.3-py3-none-any.whl |
|---|---|
| Size | 15.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
1f94a6c1b5c8975dbbd32fd1c6f167e634b0565815b3ad7c3eff796118c7d59c
|
|
BLAKE2b-256 checksum How to use checksums |
669b1592dc6abe56a784f72e8b8455f6da9ebb943c5e621182ac7aa83e7a8e7d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.14.6
|
