Skip to main content

Official Python SDK for the Scrapio

Project description

scrapio

Official Python SDK for Scrapio — fetch, crawl, search, and extract structured data from any URL.

Install

pip install scrapio-py

Requires Python 3.9 or later.

Quickstart

from scrapio import ApiClient, FetchRequest

client = ApiClient(api_key="YOUR_API_KEY")

result = client.fetch.fetch(FetchRequest(
    url="https://example.com",
    output=["markdown"],
))

print(result.outputs["markdown"])

Usage

Fetch a page

result = client.fetch.fetch(FetchRequest(
    url="https://news.ycombinator.com",
    render_js=True,
    output=["markdown"],
))

Google Search

from scrapio import GoogleSearchParams

results = client.google.search(GoogleSearchParams(
    search="best web scraping API 2025",
    country_code="us",
))
print(results.organic_results)

Amazon product

product = client.amazon.get_product("B08N5WRWNW")
print(product.title, product.price)

Walmart search

items = client.walmart.search("headphones")

YouTube transcript

video = client.youtube.get_video("dQw4w9WgXcQ")

Browser automation

result = client.interact.interact({
    "url": "https://example.com",
    "actions": [
        {"type": "click", "selector": "#login"},
        {"type": "type", "selector": "#email", "text": "user@example.com"},
    ],
})

Crawl a site

result = client.crawl.crawl({
    "seeds": ["https://docs.example.com"],
    "max_pages": 50,
})

Async jobs

from scrapio import CreateJobRequest

job = client.jobs.create(CreateJobRequest(
    job_type="fetch",
    payload={"url": "https://example.com", "output": ["markdown"]},
))
result = client.jobs.wait_for_completion(job.job_id, poll_interval=2.0, timeout=120.0)

Async client

import asyncio
from scrapio import AsyncApiClient, FetchRequest

async def main():
    async with AsyncApiClient(api_key="YOUR_API_KEY") as client:
        result = await client.fetch.fetch(FetchRequest(
            url="https://example.com",
            output=["markdown"],
        ))
        print(result.outputs["markdown"])

asyncio.run(main())

Configuration

Option Type Default Description
api_key str required Your API key
base_url str https://api.scrapio.dev Override for local/staging
timeout float 30.0 Per-request timeout (seconds)
max_retries int 3 Max retries on 429/503

Error handling

from scrapio import (
    ApiClient, FetchRequest,
    AuthError, RateLimitError, CreditsExhaustedError, ApiError,
)

try:
    client.fetch.fetch(FetchRequest(url="https://example.com"))
except AuthError:
    print("Invalid API key")
except CreditsExhaustedError:
    print("No credits remaining")
except RateLimitError:
    print("Rate limited — back off and retry")
except ApiError as e:
    print(f"API error {e.status_code}: {e}")

Links

License

MIT

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

scrapio_py-1.0.0.tar.gz (6.9 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

scrapio_py-1.0.0-py3-none-any.whl (10.2 kB view details)

Uploaded Python 3

File details

Details for the file scrapio_py-1.0.0.tar.gz.

File metadata

  • Download URL: scrapio_py-1.0.0.tar.gz
  • Upload date:
  • Size: 6.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.9.6

File hashes

Hashes for scrapio_py-1.0.0.tar.gz
Algorithm Hash digest
SHA256 28b20ff2e6a90c7a516be51a3e57f67ece2f980961ab4ca3e427e454d5ca9957
MD5 32b0f379bed0e19df4ba3073488c143c
BLAKE2b-256 9ece65e1000459539dfce42a3bb91c7eb0b15934d255758f2716af79ffe43498

See more details on using hashes here.

File details

Details for the file scrapio_py-1.0.0-py3-none-any.whl.

File metadata

  • Download URL: scrapio_py-1.0.0-py3-none-any.whl
  • Upload date:
  • Size: 10.2 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.9.6

File hashes

Hashes for scrapio_py-1.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 5d6c7894eeaf5c6e9b927e30553e2ada8d1d77452f1f67fc77c1f16281dd7f92
MD5 f28198f29350130a382b6a2093970343
BLAKE2b-256 bc5adade275e7ac4fab1a5625066ae4c6188427ff7ffc5bbae6e959f8923a6a1

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page