Skip to main content

A Wrapper around Reddit RSS feed

Project description

License PyPI - Python Version Release Downloads Last commit

Reddit RSS Reader

This is wrapper around publicly/privately available Reddit RSS feeds. It can be used to fetch content from front page, subreddit, all comments of subreddit, all comments of a certain post, comments of certain reddit user, search pages and many more. For more details about what type of RSS feed is provided by Reddit refer these links: link1 and link2.

*Note: These feeds are rate limited hence can only be used for testing purpose. For serious scrapping register your bot at apps to get client details and use it with Praw.

Installation

Install via PyPi:

pip install reddit-rss-reader

Install from master branch (if you want to try the latest features):

git clone https://github.com/lalitpagaria/reddit-rss-reader
cd reddit-rss-reader
pip install --editable .

How to use

RedditRSSReader require feed url, hence refer link to generate. For example to fetch all comments on subreddit r/wallstreetbets -

https://www.reddit.com/r/wallstreetbets/comments/.rss?sort=new

Now you can run the following example -

import pprint
from datetime import datetime, timedelta

import pytz as pytz

from reddit_rss_reader.reader import RedditRSSReader


reader = RedditRSSReader(
    url="https://www.reddit.com/r/wallstreetbets/comments/.rss?sort=new"
)

# To consider comments entered in past 5 days only
since_time = datetime.utcnow().astimezone(pytz.utc) + timedelta(days=-5)

# fetch_content will fetch all contents if no parameters are passed.
# If `after` is passed then it will fetch contents after this date
# If `since_id` is passed then it will fetch contents after this id
reviews = reader.fetch_content(
    after=since_time
)

pp = pprint.PrettyPrinter(indent=4)
for review in reviews:
    pp.pprint(review.__dict__)

Reader return RedditContent which have following information (extracted_text and image_alt_text are extracted from Reddit content via BeautifulSoup) -

@dataclass
class RedditContent:
    title: str
    link: int
    id: str
    content: str
    extracted_text: Optional[str]
    image_alt_text: Optional[str]
    updated: datetime
    author_uri: str
    author_name: str
    category: str

The output is given with UTF-8 charsets, if you are scraping non-english reddits then set the environment to use UTF -

export LANG=en_US.UTF-8
export PYTHONIOENCODING=utf-8

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for reddit-rss-reader, version 1.3.2
Filename, size File type Python version Upload date Hashes
Filename, size reddit_rss_reader-1.3.2-py3-none-any.whl (8.5 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size reddit-rss-reader-1.3.2.tar.gz (4.4 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page