webglean 0.1.0

Official Python client for the WebGlean API

webglean

Official Python client for the WebGlean API.

pip install webglean

from webglean import WebGlean

client = WebGlean(api_key="wg_your_key")  # or set WEBGLEAN_API_KEY

result = client.scrape("https://example.com")
print(result["markdown"])

WebGlean also works as a context manager (with WebGlean(...) as client:), which closes the underlying HTTP connection pool on exit.

Methods

Method	Endpoint
scrape(url, ...)	POST /v1/scrape
crawl(url, ...) / get_crawl(id) / crawl_and_wait(id, ...)	POST /v1/crawl, GET /v1/crawl/:id
extract(url, schema=, prompt=)	POST /v1/extract
map(url, ...)	POST /v1/map
search(query, ...)	POST /v1/search
create_monitor(url, ...) / list_monitors() / get_monitor(id) / delete_monitor(id)	POST/GET/DELETE /v1/monitor
batch_scrape(urls=, items=, ...) / get_batch(id) / batch_scrape_and_wait(id, ...)	POST /v1/batch/scrape, GET /v1/batch/scrape/:id

crawl_and_wait and batch_scrape_and_wait poll every 2s by default (poll_interval, timeout in seconds) until the job finishes, raising WebGleanError (status 0) on timeout.

Errors

Any non-2xx response raises WebGleanError, with .status and the message set from the API's error body.

Development

python3 -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
pytest