webextract-sdk (Python)

Thin client for the WebExtract API: a public web page in, clean Markdown plus fixed-schema JSON out (article metadata, product and price, tables). It's built for AI agents.

pip install webextract-sdk            # core (httpx only)
pip install "webextract-sdk[x402]"    # + the official x402 SDK, for pay-per-call

The PyPI name is webextract-sdk; the import name is webextract (from webextract import WebExtract).

Billing: pick one

This package never handles private keys. With x402, signing happens inside the client you build with the official x402 SDK.

from webextract import WebExtract, PaymentRequiredError, TargetFetchError

with WebExtract() as we:                      # free tier
    r = we.extract("https://web-scraping.dev/product/1")
    print(r.data["product"], r.meta.free_remaining)

x402 (async, official SDK)

import asyncio, os
from eth_account import Account
from x402 import x402Client
from x402.http.clients import x402HttpxClient
from x402.mechanisms.evm.exact import ExactEvmScheme
from webextract import AsyncWebExtract

async def main():
    signer = Account.from_key(os.environ["X402_PRIVATE_KEY"])    # your key, your process
    xc = x402Client(); xc.register("eip155:8453", ExactEvmScheme(signer=signer))
    async with x402HttpxClient(xc, timeout=60) as pay:
        we = AsyncWebExtract(payment_client=pay)                 # billing="x402"
        r = await we.markdown("https://example.com")
        print(r.meta.payment_response)                           # settlement receipt (tx hash)
asyncio.run(main())

WEBEXTRACT_API_KEY: if api_key isn't passed, the client reads this environment variable. An explicit api_key always wins. The variable is ignored when payment_client is given or billing is anything other than "key", so it never causes the "api_key + payment_client" error.

settlement_retries (default 1): a 402 settlement_failed (payment verified, not settled, not charged) is retried once, and the payment client signs a fresh payment. Other errors are never retried.

For a sync client, pass the requests.Session from x402.http.clients.x402_requests(xc) as payment_client=. The x402 SDK has no sync httpx client.

Errors

Every error is a WebExtractError with status, code, message, charged, retryable, body.

Response fields

The responses are the API JSON as-is (schema_version "1.0"). Fields added on 2026-10-21 (content_source, product.price_detail, product_unresolved) may be missing on older deployments, so read them with .get(). A missing price is never reported as 0. price_detail.unresolved.reason says why it's missing.

License: MIT © TAINCHENG, L.L.C.