scrapenest 0.4.0

Official Python client for the ScrapeNest API — web scraping, search, AI answers, screenshots, PDFs, Google Maps reviews, and transcription.

scrapenest

Official Python client for the ScrapeNest API — web scraping, search, AI answers, screenshots, PDFs, Google Maps reviews, gas prices, and audio transcription.

• Docs: https://scrapenest.dev/docs
• Get an API key: https://scrapenest.dev

Install

pip install scrapenest

Requires Python 3.9+.

Quick start

from scrapenest import Client

client = Client("sn_your_key_here")

result = client.search("python web scraping", num_results=5)
for r in result.results:
    print(r.title, r.url)

Every method returns a typed dataclass. Pass include_usage=True (where supported) to get a usage block with the credits charged.

Search

# SERP results. depth = "fast" | "balanced" | "thorough"
hits = client.search("openai news", depth="balanced", num_results=10)

# Search + fetch & clean the top pages
deep = client.search_deep("what is rust lang", fetch_top=3)
for page in deep.pages:
    print(page.url, len(page.text or ""))

# Cited answer synthesized from live sources
answer = client.search_answer("who wrote the odyssey", max_sources=3)
print(answer.answer)
for c in answer.citations:
    print(c.index, c.url)

Scrape a page

page = client.scrape_url(
    "https://example.com",
    render="auto",          # auto | direct | browser | stealth
    extract=True,
    return_markdown=True,
)
print(page.title)
print(page.markdown[:500])

# AI extraction
data = client.scrape_url("https://example.com/product", ai_query="Return the product name and price")
print(data.ai_extract)

Screenshots & PDF

shot = client.screenshot("https://example.com", full_page=True, format="png")
with open("page.png", "wb") as f:
    import base64
    f.write(base64.b64decode(shot.image_base64))

doc = client.pdf("https://example.com", format="A4", landscape=False)
with open("page.pdf", "wb") as f:
    import base64
    f.write(base64.b64decode(doc.pdf_base64))

Google Maps reviews

Pass a place URL, a 0x…:0x… FID, or a ChIJ… place id. Use place_url to bypass FID-built URLs for tricky listings.

res = client.maps_reviews("https://www.google.com/maps/place/...", max_reviews=50, sort="newest")
print(res.name, res.rating, res.review_count)
for rev in res.reviews:
    print(rev.author_name, rev.rating, rev.text)

# Large pulls run as async jobs
run = client.maps_reviews_async("ChIJN1t_tDeuEmsRUsoyG83frY4", max_reviews=2000)
final = client.wait_for_run(run.run_id)
print(f"{len(final.result['reviews'])} reviews fetched")
# Need to stop a long job early? client.cancel_run(run.run_id)

Gas prices

gas = client.gas_prices("Los Angeles, CA", grade="regular", limit=10)
for s in gas.stations:
    print(s.brand, s.address, s.price)
if gas.region_stats:
    print(gas.region_stats.region, gas.region_stats.average_price)

Translate & transcribe

tr = client.translate("Hello, how are you?", "es")
print(tr.translated)

audio = client.audio_transcript("https://www.youtube.com/watch?v=...")
print(audio.text)

Async client

import asyncio
from scrapenest import AsyncClient

async def main():
    async with AsyncClient("sn_your_key_here") as client:
        result = await client.search("python web scraping")
        print(result.results[0].title)

asyncio.run(main())

AsyncClient exposes the same methods as Client.

Error handling

from scrapenest import RateLimitError, AuthenticationError, PaymentRequiredError

try:
    result = client.search("test")
except RateLimitError as e:
    print(f"Rate limited. Retry after {e.retry_after}s")
except PaymentRequiredError:
    print("Out of credits")
except AuthenticationError:
    print("Invalid API key")

License

MIT