Skip to main content

WebCrawler API Python SDK

A Python SDK for interacting with the WebCrawlerAPI.

Get your API key at dash.webcrawlerapi.com/access.

Installation

pip install webcrawlerapi

Crawling

from webcrawlerapi import WebCrawlerAPI

client = WebCrawlerAPI(api_key="your_api_key")

job = client.crawl(
    url="https://example.com",
    output_formats=["markdown"],
    items_limit=10,
)

for item in job.job_items:
    if item.status == "done":
        print(item.original_url)
        print(item.content)

Scraping

from webcrawlerapi import WebCrawlerAPI

client = WebCrawlerAPI(api_key="your_api_key")

result = client.scrape(
    url="https://example.com",
    output_formats=["markdown"],
)

if result.success:
    print(result.markdown)
else:
    print(f"Error: {result.error_code} — {result.error_message}")

Documentation

Requirements

  • Python 3.6+
  • requests >= 2.25.0

License

MIT License

Metadata

Release files for webcrawlerapi 2.1.6

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for webcrawlerapi 2.1.6
File Size Uploaded Provenance Actions
webcrawlerapi-2.1.6.tar.gz 14.6 kB Without provenance Details for webcrawlerapi-2.1.6.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for webcrawlerapi 2.1.6
File Interpreter ABI Platform Actions
webcrawlerapi-2.1.6-py3-none-any.whl Python 3 none any Details for webcrawlerapi-2.1.6-py3-none-any.whl

Total release size: 30.2 kB

Release files / webcrawlerapi-2.1.6.tar.gz

Download URL webcrawlerapi-2.1.6.tar.gz
Size 14.6 kB
Tags Source
SHA-256 checksum
How to use checksums
f74ecad01e88e3976acb74a1675275ffaa64e0de260a045dd464cfbef97bca4f
BLAKE2b-256 checksum
How to use checksums
8d9b9df2f837eb9b12a5a4b10b05e412b864f9ab7c1df9d70de8bda04d002cdd
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.14.8

Release files / webcrawlerapi-2.1.6-py3-none-any.whl

Download URL webcrawlerapi-2.1.6-py3-none-any.whl
Size 15.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
1026047f82b58c53bf0abf092b62841541997622adc696906738c57ac92ba620
BLAKE2b-256 checksum
How to use checksums
843bb9dc08b12cee367e2a779d7b262ce4ce2ce83f1ede0bced70b436d348016
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.14.8

Security for webcrawlerapi 2.1.6

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page