gnews-decoder

Turn news.google.com/rss/articles/CBMi... links into the publisher URLs behind them — in one call for the whole list, not one call per article.

from gnews_decoder import from_rss, decode, decode_many

for item in from_rss("artificial intelligence"):
    print(item["url"])      # https://www.economist.com/...

decode("https://news.google.com/rss/articles/CBMi...")
decode_many(urls)           # {google_url: publisher_url or None}

pip install gnews-decoder

Measured against a live feed on 19 September 2026: 50 of 50 resolved in 6.8 seconds, using 50 signature requests and one decode call.

Why this exists

Google News links do not contain the destination. Until late 2024 the id after /articles/ was base64 of a small protobuf with the URL inside, and every library in this space decoded it offline with zero requests.

That format is gone. Of 30 ids taken from a live feed on 19 September 2026, 0 decoded offline and all 30 needed Google's batchexecute RPC. This is what "base 64 deprecation" — the most-discussed open issue on the abandoned pygooglenews — actually means, and why so much code in this area quietly returns nothing.

Two other walls come with it, and this package handles both:

The consent interstitial. Without a recorded consent choice every article URL 302s to consent.google.com, and the page that comes back carries no signature. That is the "returns a URL about Google Consent Mode" report that turns up in every library's issue tracker. Measured: 0 of 6 signatures without the cookie, 6 of 6 with it. Pass consent_cookie= to supply your own.

Rate limiting. The obvious implementation issues one RPC per article and Google starts answering 429 — which is why the alternatives' release notes are a history of retry delays and proxy support. batchexecute takes every article in a single POST; 50 resolve in about 0.8s of RPC time. That is the difference between 100 requests and 51.

One more thing worth knowing, because it breaks scrapers: the HTML search page at news.google.com/search answers 429 almost immediately, while news.google.com/rss/search answers 200. from_rss uses the feed.

What it does not do

It does not give you the article text. The Google News feed carries headlines, links and timestamps — no body, not even a summary. Once you have the publisher URL you still have to fetch and extract it, and a meaningful share of publishers will not serve you: some want a subscription, some refuse the request outright. That is a separate and much harder problem, and this package does not pretend to solve it.

It also will not get you past the ~100-article ceiling per feed query, because that limit is Google's and no amount of decoding changes it.

API

Keyword arguments: ceid (default "US:en"), concurrency (16), timeout (20.0), consent_cookie, session.

If you need the text too

This package is the free half of a problem we spend our time on. If you want the article body, at scale, with the paywalled and blocked cases labelled rather than silently dropped, that is hawkcrawl — and the measurements behind all of the above, including the feed parameters Google does not document, are written up at hawkcrawl.com/google-news-rss.

You do not need it to use this. This works on its own and always will.

Licence

MIT.