ScrapingIsNotACrime Python SDK

Official Python SDK for the ScrapingIsNotACrime public data API — typed access to Instagram, TikTok, YouTube, App Store, GitHub, Hacker News, Bluesky, Twitch and Linktree, with sync and async clients.

Install

pip install scrapingisnotacrime

Requires Python 3.11+.

Quick start

Sync:

from scrapingisnotacrime import ScrapingIsNotACrime

with ScrapingIsNotACrime() as client:
    profile = client.instagram.profile("nasa")
    print(profile["username"], profile["followers"])

Async:

import asyncio

from scrapingisnotacrime import AsyncScrapingIsNotACrime


async def main() -> None:
    async with AsyncScrapingIsNotACrime() as client:
        profile = await client.instagram.profile("nasa")
        print(profile["username"], profile["followers"])


asyncio.run(main())

Both clients read the API key from the SCRAPINGISNOTACRIME_API_KEY environment variable, or take it as api_key=. Get a key at scrapingisnotacrime.com/dashboard/api-keys. Keys start with sinac_.

More runnable examples are in examples/: profile.py, paginate_followers.py and async_posts.py.

Configuration

ScrapingIsNotACrime(
    api_key="sinac_...",
    base_url="https://api.scrapingisnotacrime.com/v1",
    timeout=30.0,
    max_retries=2,
    http_client=None,
)

AsyncScrapingIsNotACrime takes the same arguments (http_client accepts an httpx.AsyncClient instead).

AsyncScrapingIsNotACrime is built on httpx.AsyncClient and requires asyncio; it does not run under trio.

Methods

Every method returns the response envelope's data, typed as a TypedDict. Methods marked → Page / → AsyncPage return a lazily-iterable page (see Pagination). Optional arguments are keyword-only.

The async client (AsyncScrapingIsNotACrime) exposes the same namespaces and methods as awaitable coroutines.

Path arguments are validated before any request: an empty string, ".", "..", a bool, or a non-finite number (nan, inf) raises ValueError. On the async client this is raised by the awaited coroutine, never at call time.

Pagination

Methods marked → Page return a Page[T] (or AsyncPage[T] on the async client):

class Page(Generic[T]):
    data: Any  # the full response of this page
    items: list[T]  # this page's items
    has_more: bool
    next_cursor: str | None  # cursor endpoints
    next_page: int | None  # page-number endpoints

    def next(self) -> Page[T] | None: ...  # the following page, or None
    def __iter__(self) -> Iterator[T]: ...  # every item, fetching further pages lazily

Iterate every item, fetching further pages lazily as they're needed. Iterating fetches every remaining page; each page is one billed request, so bound the loop:

page = client.github.followers("torvalds", limit=100)
for count, user in enumerate(page, start=1):
    print(user["username"])
    if count >= 100:
        break  # stop early; no further pages are fetched

Async, with async for:

page = await client.bluesky.posts("bsky.app", limit=25)
count = 0
async for post in page:
    print(post)
    count += 1
    if count >= 100:
        break  # stop early; no further pages are fetched

page.data gives you the untouched response of the current page, so fields such as total stay reachable. Breaking out of the loop early stops fetching — no further pages are requested once you break.

Types

Responses are plain dicts typed with TypedDict, importable from scrapingisnotacrime.types:

from scrapingisnotacrime.types import InstagramProfile

Field names are kept exactly as the API sends them (snake_case, or camelCase for the App Store endpoints). Fields the upstream platforms routinely leave empty — descriptions, bios, captions, languages, last-broadcast info — are typed NotRequired[T | None], so they may be missing from the dict entirely or present as None; check before use.

Errors

Every failure raises a subclass of ScrapingIsNotACrimeError, carrying status (the HTTP status, or None for network errors), message and request_id (when the API returns one).

ConnectionError here is the SDK's own class — import it from scrapingisnotacrime, not the builtin ConnectionError.

from scrapingisnotacrime import (
    NotFoundError,
    QuotaExceededError,
    RateLimitError,
    ScrapingIsNotACrime,
)

client = ScrapingIsNotACrime(max_retries=3)

try:
    client.tiktok.profile("this-user-does-not-exist-123")
except NotFoundError:
    print("No such profile.")
except QuotaExceededError as error:
    print("Out of credits:", error.message)
except RateLimitError:
    print("TikTok is rate limiting; try again later.")

Retries

RateLimitError (429), UpstreamError (502) and ConnectionError (network failure or timeout) are retried automatically, up to max_retries additional attempts (default 2). These failures don't consume credits, so retrying them costs you nothing.

The delay before each retry uses the Retry-After header when the API sends one (seconds or an HTTP date); otherwise it's exponential backoff with full jitter, starting at 500 ms and doubling per attempt. Every wait is capped at 10 seconds. Set max_retries=0 to disable retries entirely.

Releases and changelog

Every merge to main is released automatically: the version comes from the commit messages since the last release, following Conventional Commits.

The pipeline tags vX.Y.Z, publishes the GitHub Release with the notes, and publishes to PyPI through Trusted Publishing. The changelog is the Releases page; the version is resolved from git tags at build time (hatch-vcs), so no version number is written into the repository.

Merge PRs with a merge commit or rebase so each Conventional Commit is analysed; if you squash, the PR title must be a Conventional Commit (e.g. feat: ...).

If the publish job fails after a release was tagged, use Re-run failed jobs on that run, or run the Release workflow manually with the tag.

Links

• Docs: https://scrapingisnotacrime.com/docs
• Pricing: https://scrapingisnotacrime.com/#pricing
• Releases: https://github.com/ScrapingIsNotACrime/sdk-python/releases
• Node.js SDK: https://github.com/ScrapingIsNotACrime/sdk-nodejs
• MCP server: https://github.com/ScrapingIsNotACrime/mcp
• License: MIT