Web scraper for subreddits
Project description
sreddit
A simple tool for scraping information from subreddits.
Installation
To install, you can either Download the Pypi Package or,
pip install sreddit
To upgrade to the latest version,
pip install --upgrade sreddit
Usage
srtitles
Gets all unique titles from a subreddit.
from sreddit import SubRedditTitles
scraper = SubRedditTitles(subreddit="subreddit_name")
scraper.run()
srbodies
Gets all unique post bodies (i.e. descriptions) from a subreddit.
from sreddit import SubRedditBodies
scraper = SubRedditBodies(subreddit="subreddit_name")
scraper.run()
Optional Arguments
Argument | What it Does |
---|---|
keywords | Only includes content that has one or more of these keywords |
show_progess | Whether or not to show scraping progress (i.e. number of titles found) in the terminal |
make_db | If a database of the content found should be created after scraping |
db_name | The name of the database to be created--must end in .db |
scroll_time | Time to wait between scrolling down the page and finding elements. |
FAQs
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
sreddit-1.0.3.tar.gz
(5.2 kB
view details)
Built Distribution
File details
Details for the file sreddit-1.0.3.tar.gz
.
File metadata
- Download URL: sreddit-1.0.3.tar.gz
- Upload date:
- Size: 5.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.9.17
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | f160ae4a8ce0be2ed34bd542e092b85d0a0ce8100d13633d2e7b9df64824d6fb |
|
MD5 | 5f8fc226323dde57b7fe6a23f65544e5 |
|
BLAKE2b-256 | 3eb8f8b554cc25f9b60edf180ad44a43df59cfdd5b6e8033536ddb839ca1b679 |
File details
Details for the file sreddit-1.0.3-py3-none-any.whl
.
File metadata
- Download URL: sreddit-1.0.3-py3-none-any.whl
- Upload date:
- Size: 7.3 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.9.17
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 79465ea2b4a8708eaab0914b1a6ab89104e92409f35dcbc0363354de18d193ef |
|
MD5 | 54ee8a6c01f9ad45daf1b9eff32558ca |
|
BLAKE2b-256 | 99adad33da1ff65b43aeca9cd100a3c62597708cb2d79329552ecfc6723aa04d |