gemini-scraper

A Python client for the Gemini API endpoint on ScrapingBee. Send a prompt to Google Gemini, get back the answer as text, as markdown, and with citations when Gemini attaches them. No headless browser to run, no Google account to manage.

Verified against the live endpoint on 2026-08-25: GET https://app.scrapingbee.com/api/v1/gemini, 15 credits per successful call.

Install

pip install gemini-scraper

Python 3.8+ and requests. You need a ScrapingBee key from app.scrapingbee.com, which starts with 1,000 free credits.

One prompt

from gemini_scraper import GeminiScraper

scraper = GeminiScraper("YOUR-API-KEY")
answer = scraper.ask("Best programming languages for data science")

print(answer.markdown)
print(answer.citations)
print(answer.cost, answer.request_id)

What the Gemini scraper returns

The endpoint answers with five fields. These are the real keys observed on a live call, not a guess at the shape:

Citations are returned when the prompt produces them. Treat the list as best effort rather than a guarantee, and check len(answer.citations) before relying on it.

Localising an answer

Gemini answers differently depending on where the request comes from. Anything involving pricing, availability, regulation or local providers is worth running per market:

for market in ["us", "gb", "de", "fr"]:
    answer = scraper.ask("best project management software", country_code=market)
    print(market, len(answer.citations), answer.text[:120])

Running a prompt set

Brand visibility work means the same prompt list on a schedule. ask_many keeps going when a single prompt fails, returning None in that slot rather than losing the batch:

prompts = [
    "best web scraping api",
    "how do I scrape a javascript heavy site",
    "cheapest way to collect serp data",
]

for prompt, answer in zip(prompts, scraper.ask_many(prompts, country_code="us")):
    if answer is None:
        print("failed:", prompt)
        continue
    print(prompt, "->", answer.text[:100])

At 15 credits per call, a 100-prompt set costs 1,500 credits per run. Check the balance first:

print(scraper.usage())
# {'max_api_credit': ..., 'used_api_credit': ..., 'max_concurrency': ..., 'current_concurrency': ...}

usage() is free and capped at 6 calls per minute.

Tagging requests

tag attaches your own identifier so a call can be found later in the dashboard, which matters when several jobs share one key:

scraper.ask("best crm for small teams", tag="visibility-run-2026-08")

Errors

GeminiScraperError carries status_code, body and request_id. Quote the request id to support. HTTP 500 responses are not charged credits, so a retry costs nothing:

from gemini_scraper import GeminiScraper, GeminiScraperError

try:
    answer = scraper.ask("...")
except GeminiScraperError as error:
    print(error.status_code, error.request_id)

Cost

Plan tiers at scrapingbee.com/pricing.

Related scrapers

The same account also reaches ChatGPT, Google search, Google AI Mode, Google News and Perplexity, which is the usual set for tracking how a brand shows up across AI answers.

Scope

Public, pre-login content only. Scraping behind login credentials is prohibited by the ScrapingBee terms. Keep your API key out of shared environments, including AI coding assistants.

More: Gemini feature page . documentation . code in eight languages

MIT licensed.