google-news-scraper-api

A news scraper API for Python, built on ScrapingBee's Google News scraper. It turns a query into structured articles with title, source, domain, snippet and timestamp, so you never touch Google's markup.

Verified live on 2026-08-25 against GET /api/v1/google with search_type=news. Calls in testing charged 10 credits.

Install

pip install google-news-scraper-api

Scrape Google News results

from google_news_scraper_api import GoogleNewsScraper

scraper = GoogleNewsScraper("YOUR-API-KEY")

for article in scraper.search("openai", country_code="us"):
    print(article.date, article.source, article.title)

Get a key at app.scrapingbee.com; new accounts include 1,000 free credits.

Article fields

Every result carries the nine keys the API actually returns:

article.as_dict() gives you all nine at once, which is what you want before writing to a database or a dataframe.

Paging without waste

paginate walks result pages and stops the moment one comes back empty, so an optimistic page count does not spend credits on nothing:

articles = list(scraper.paginate("climate policy", pages=5, country_code="us"))
print(len(articles))

Deduplicating a monitor

Syndicated stories repeat across pages and across queries. Counting them twice makes a media monitor useless, so dedupe by link before you store anything:

raw = []
for query in ["openai", "anthropic", "mistral ai"]:
    raw.extend(scraper.paginate(query, pages=2, country_code="us"))

unique = GoogleNewsScraper.deduplicate(raw)
print(len(raw), "->", len(unique))

Query controls worth knowing

scraper.search("acme corp", country_code="gb", date_range="d")   # recent window
scraper.search("scrapingbee", nfpr=True)                         # no spelling "correction"

nfpr=True matters more than it looks. Google rewrites unfamiliar brand names to something it considers correct, which quietly turns a brand monitor into a monitor for a different word.

One documented limit: search_type=news is not available with device="mobile".

Raw payload

When you want meta_data as well as the articles:

payload = scraper.search_raw("openai", country_code="us")
print(payload["meta_data"])
# {'url': 'https://www.google.com/search?q=openai&hl=en&gl=us&tbm=nws', 'number_of_page': 10, ...}

Cost

Retrying a 500 is free. Check the balance with scraper.usage() before a scheduled sweep. Tiers on the pricing page.

Other Google surfaces

The same endpoint switches surface with search_type, so the same key also reaches Google search, Shopping, Images, Lens, AI Mode and Ads. For a lighter alternative there is Fast Search, and for an RSS-shaped feed see the Google News RSS API.

Scope

Public, pre-login content only. Scraping behind login credentials is prohibited by the ScrapingBee terms. Keep API keys out of AI coding assistants.

MIT licensed. Repository . API reference