MrScraper Python SDK

An async Python client for MrScraper content fetching, Google SERP, and account status. Requires Python 3.9+.

Install and authenticate

python -m pip install -e .
export MRSCRAPER_API_TOKEN="YOUR_TOKEN"

These commands install the current checkout. To install version 0.6.0 from PyPI, run python -m pip install --upgrade mrscraper-sdk==0.6.0. The client reads MRSCRAPER_API_TOKEN automatically. You can also pass token="YOUR_TOKEN" to MrScraper. Pass http_client= if you want to reuse your own httpx.AsyncClient; otherwise the SDK creates one per request. All three methods are async and must be awaited.

Version 0.6.0

Version 0.6.0 restores await client.account() alongside scraper_api and web_search. Account status includes identity, subscription, token quota, and rate limits, with API keys and billing identifiers excluded.

Migrating to 0.5.0

Version 0.5.0 renames two public methods. Update existing calls as follows:

The previous method names have been removed. Arguments and response formats are unchanged from 0.4.0.

Scraper API

import asyncio
from mrscraper import MrScraper

async def main():
    client = MrScraper()
    result = await client.scraper_api("https://example.com")
    print(result["data"])

asyncio.run(main())

scraper_api sends a POST request to https://api.mrscraper.com. The JSON body contains the target URL, maxRetries, and optional tokenCap. Page options and the token are query parameters. The return value has status_code, data, and headers keys. The SDK parses JSON responses into Python values and returns other responses as text. Depending on the requested options, data may contain HTML, Markdown, a screenshot, or a combination; inspect the returned value before assuming it is an HTML string.

To request HTML, Markdown, and a screenshot together:

import asyncio
from mrscraper import MrScraper

async def main():
    client = MrScraper()
    result = await client.scraper_api(
        "https://example.com",
        html=True,
        markdown=True,
        screenshot=True,
        super=True,
    )
    print(result["data"])

asyncio.run(main())

Setting screenshot=True also enables browser rendering, even if browser_rendering=False is passed.

Web search

import asyncio
from mrscraper import MrScraper

async def main():
    client = MrScraper()
    result = await client.web_search(
        "python web scraping",
        raw=False,
        region="us",
        language="en",
        page=1,
        format="json",
    )
    print(result["data"])

asyncio.run(main())

web_search sends a POST request to https://sync.scraper.mrscraper.com/api/google/serp/v2/sync. The first argument accepts query text or a full Google search URL; for a URL, the client extracts q, gl, hl, and start. Explicit options override values from the URL. The return value has status_code, data, and headers keys. raw=True is the default and requests HTML; set raw=False to use format.

Account status

import asyncio
from mrscraper import MrScraper

async def main():
    result = await MrScraper().account(timeout=30)
    print(result["data"]["subscription_status"])
    print(result["data"]["token_remaining"])
    print(result["data"]["user"]["email"])

asyncio.run(main())

account() sends GET https://api.app.mrscraper.com/api/v1/subscription-accounts using the client's token in Bearer and x-api-token headers. The timeout defaults to 30 seconds and must be positive. The response has status_code, data, and headers keys. data contains subscription_status, enterprise, token_usage, token_limit, token_remaining, usage_percent, rate_limit, rate_ttl, auto_renew, ends_at, and user (name, email, verified). Remaining tokens are clamped to zero when usage exceeds the limit. API keys, billing identifiers, and sensitive response headers are excluded.

Smoke test this checkout

file.py runs one live request per invocation. Set your token first, then run:

export MRSCRAPER_API_TOKEN="YOUR_MRSCRAPER_API_TOKEN"
python file.py account --timeout 30
python file.py fetch-content --url "https://example.com" --super
python file.py google-serp --query "site:example.com" --format json

The account command calls account() from the local SDK checkout. A successful request prints HTTP 200 followed by the account summary, including subscription status, token usage, remaining tokens, and user details. API or timeout errors print ERROR and exit with a nonzero status. Use your MrScraper API token from the app.

The fetch-content command calls scraper_api() and enables screenshot=true and warmStart=true by default. Pass --super to enable real device routing; the printed request URL should contain super=true. The google-serp command calls web_search() and requests JSON by default.

Use python file.py --help for the available options. Add --save-dir manual_results to save a full JSON response.

Publishing a release

Publishing is manual. Choose a version that has not been published to PyPI and set it in both pyproject.toml and src/mrscraper/__init__.py. This checkout is prepared for 0.6.0. From the repository root, run:

python -m pip install build twine pytest pytest-asyncio
python -m pip install -e .
python -m pytest -m "not integration" -q
python -m build --outdir dist/manual-0.6.0
ls dist/manual-0.6.0
python -m twine check dist/manual-0.6.0/mrscraper_sdk-0.6.0*
python -m twine upload --username __token__ dist/manual-0.6.0/mrscraper_sdk-0.6.0*

Before uploading, confirm that ls shows only the intended 0.6.0 files. Twine prompts for a PyPI API token as the password. Do not upload older files from dist/.

Errors

MrScraperError is the base exception. AuthenticationError indicates a missing or invalid token, APIError carries an HTTP status_code, and NetworkError indicates a connection or timeout failure.

Compliance & Legal Risk

Scraping login-protected pages can carry legal and compliance risks. Check the target site's terms and applicable law before automating access.

License

MIT © MrScraper