Installation

The library is available on PyPI:

pip install feed_seeker

The library requires Python 3.5+.

Quickstart

By default, the library uses requests to grab html and inspect it and find the most likely feed url:

from feed_seeker import find_feed_url

>>> find_feed_url('https://github.com/mitmedialab/feed_seeker')
'https://github.com/mitmedialab/feed_seeker/commits/master.atom'

To do a more thorough search, use generate_feed_urls, which returns more likely candidates first.

from feed_seeker import generate_feed_urls

>>> for url in generate_feed_urls('https://xkcd.com'):
...     print(url)
...
https://xkcd.com/atom.xml
https://xkcd.com/rss.xml

For the most thorough search, add a spider argument to do depth-first spidering of urls on the same hostname. Note the below call takes nearly four minutes, compared to 0.5 seconds for find_feed_url.

>>> for url in generate_feed_urls('https://github.com/mitmedialab/feed_seeker', spider=1):
...     print(url)
...
https://github.com/mitmedialab/feed_seeker/commits/master.atom,
https://github.com/mitmedialab/feed_seeker/commits/95cf320796c487df8b70f9c42281d8f26452cc31.atom,
https://github.com/mitmedialab/feed_seeker/commits/3e93490cb91f7652325c2fe41ef29a5be4558d6a.atom,
https://github.com/mitmedialab/feed_seeker/commits/659311b8853c4c4a67e3b4bc67a78461d825a064.atom,
https://github.com/mitmedialab/feed_seeker/commits/a8f7b86eac2cedd9209ac5d2ddcceb293d2404c9.atom,
https://github.com/index.atom,
https://github.com/articles.atom,
https://github.com/dfm/feedfinder2/commits/master.atom,
https://github.com/blog.atom,
https://github.com/blog/all.atom,
https://github.com/blog/broadcasts.atom,
https://github.com/ColCarroll.atom

New Feedly Support

Search Feedly to find relevant feeds to a specific site:

from feed_seeker import find_feedly_feeds

>>> for url in find_feedly_feeds('https://www.nytimes.com'):
...     print(url)
...

http://www.nytimes.com/services/xml/rss/nyt/HomePage.xml
http://www.nytimes.com/services/xml/rss/nyt/HomePage.xml
http://www.nytimes.com/services/xml/rss/nyt/Science.xml
http://www.nytimes.com/services/xml/rss/nyt/Travel.xml
http://www.nytimes.com/services/xml/rss/nyt/Technology.xml
http://www.nytimes.com/services/xml/rss/nyt/Business.xml

In a hurry?

If you have a long list of urls, you might want to set a timeout with max_time:

>>> for url in ('https://httpstat.us/200?sleep=5000', 'https://github.com/mitmedialab/feed_seeker'):
   ...     try:
   ...         print('found feed:\t{}'.format(find_feed_url(url, max_time=3)))
   ...     except TimeoutError:
   ...         print('skipping {}'.format(url))
   skipping https://httpstat.us/200?sleep=5000
   found feed:  https://github.com/mitmedialab/feed_seeker/commits/master.atom