Skip to main content

Web scraper for subreddits

Project description

sreddit

A simple tool for scraping information from subreddits.


Installation

To install, you can either Download the Pypi Package or,

pip install sreddit

To upgrade to the latest version,

pip install --upgrade sreddit

Usage

srtitles

Gets all unique titles from a subreddit.

from sreddit import SubRedditTitles

scraper = SubRedditTitles(subreddit="subreddit_name")
scraper.run()

srbodies

Gets all unique post bodies (i.e. descriptions) from a subreddit.

from sreddit import SubRedditBodies

scraper = SubRedditBodies(subreddit="subreddit_name")
scraper.run()

Optional Arguments


Argument What it Does
keywords Only includes content that has one or more of these keywords
show_progess Whether or not to show scraping progress (i.e. number of titles found) in the terminal
make_db If a database of the content found should be created after scraping
db_name The name of the database to be created--must end in .db
scroll_time Time to wait between scrolling down the page and finding elements.

FAQs

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

sreddit-1.0.3.tar.gz (5.2 kB view hashes)

Uploaded Source

Built Distribution

sreddit-1.0.3-py3-none-any.whl (7.3 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page