reddipy 1.0.0

Scrape Reddit posts, comments, and user activity — no API key needed. Python SDK + CLI tool.

Reddipy

Scrape Reddit posts, comments, and user activity. Python SDK + CLI.

Built on Reddit's public JSON endpoints. Two interfaces: Python SDK and CLI.

---

Features

• Search Reddit by keyword with sort, time, and subreddit filters
• Scrape posts with full comment trees, scores, and metadata
• Subreddit browsing with hot, new, top, rising, and controversial sorting
• User profiles with post/comment activity filtering
• Export to JSON, CSV, or rich console tables
• Auto-pagination to collect beyond the 100-per-request limit
• Proxy rotation with single proxy, proxy file, and dead proxy cooldown
• Rate limiting with configurable delay and exponential backoff
• Realistic headers matching a real Chrome browser fingerprint
• Lightweight with only requests and rich as dependencies

---

Installation

Recommended to install using pip

pip install reddipy

For SOCKS5 proxy support:

pip install "reddipy[socks]"

Requirements: Python 3.10+

Authentication

Reddipy requires a reddit_session cookie from your browser. This is the only cookie needed.

Step 1: Get the Cookie

1. Open reddit.com in your browser and log in
2. Open DevTools (F12) > Application tab > Cookies > https://www.reddit.com
3. Find the cookie named reddit_session
4. Copy its Value (a long JWT string starting with eyJ...)

Step 2: Configure (One-Time)

SDK:

from reddipy import Reddit

Reddit.configure(cookie="eyJhbGciOiJS...")

CLI:

reddipy configure "eyJhbGciOiJS..."

The cookie is saved to ~/.reddipy/config.json. You only need to do this once. All future SDK and CLI calls will use it automatically.

Auth Management

Reddit.configure(cookie="...")   # save cookie
Reddit.is_configured()           # check if cookie exists (returns True/False)
Reddit.logout()                  # remove stored cookie

reddipy configure "cookie"      # save
reddipy logout                  # remove

Override Stored Cookie

If you need to use a different cookie for a single session:

reddit = Reddit(cookie="different_cookie_here")

reddipy search "python" --cookie "different_cookie_here"

---

Python SDK

Initialize

from reddipy import Reddit

reddit = Reddit()

All methods below assume you have already configured authentication.

Search

Search Reddit for posts matching a keyword.

# Basic search
posts = reddit.search("machine learning", limit=10)
for post in posts:
    print(f"[{post.score}] {post.title}")
    print(f"  r/{post.subreddit} by {post.author}")

Search within a specific subreddit:

posts = reddit.search("async", subreddit="python", limit=10)

All parameters:

Parameter	Type	Default	Description
query	str	required	Search keywords
limit	int	25	Max results (up to ~1000)
sort	str	"relevance"	relevance, hot, new, top, comments
time_filter	str	"all"	day, week, month, year, all
subreddit	str	None	Scope search to a subreddit

Returns: list[Post]

Scrape a Post

Extract a single post with its full comment tree.

post, comments = reddit.post("https://www.reddit.com/r/python/comments/abc123/title/")

# Post data
print(post.title)
print(post.author)
print(post.score)
print(post.selftext)       # post body
print(post.flair)           # post flair (or None)
print(post.num_comments)
print(post.upvote_ratio)

# Comments
print(f"{len(comments)} top-level comments loaded")

for comment in comments[:5]:
    print(f"[{comment.score}] {comment.author}: {comment.body[:80]}")
    print(f"  depth: {comment.depth}, is_submitter: {comment.is_submitter}")

Accepts any Reddit post URL format:

# All of these work
reddit.post("https://www.reddit.com/r/python/comments/abc123/title/")
reddit.post("https://old.reddit.com/r/python/comments/abc123/")
reddit.post("https://reddit.com/r/python/comments/abc123/title")
reddit.post("/r/python/comments/abc123/title/")   # bare permalink

Returns: tuple[Post, list[Comment]]

Subreddit Posts

Fetch posts from a subreddit with sort and time filters.

# Hot posts (default)
posts = reddit.subreddit("python")

# Top posts this week
posts = reddit.subreddit("python", sort="top", time_filter="week", limit=50)

# New posts
posts = reddit.subreddit("datascience", sort="new", limit=20)

# Rising posts
posts = reddit.subreddit("technology", sort="rising", limit=10)

for post in posts:
    print(f"[{post.score:>5}] {post.title}")
    print(f"        {post.num_comments} comments | r/{post.subreddit}")

All parameters:

Parameter	Type	Default	Description
name	str	required	Subreddit name (without r/)
sort	str	"hot"	hot, new, top, rising, controversial
time_filter	str	"all"	day, week, month, year, all (for top/controversial)
limit	int	25	Max posts

Returns: list[Post]

User Activity

Fetch a Reddit user's recent posts and comments.

# All activity
activity = reddit.user("spez", limit=10)

for item in activity:
    print(f"[{item.type}] r/{item.subreddit} ({item.score} pts)")
    if item.type == "post":
        print(f"  {item.title}")
    else:
        print(f"  {item.body[:80]}")

Filter by activity type:

# Posts only
posts = reddit.user("spez", limit=10, activity_type="posts")

# Comments only
comments = reddit.user("spez", limit=10, activity_type="comments")

# Both (default)
all_activity = reddit.user("spez", limit=10, activity_type="all")

All parameters:

Parameter	Type	Default	Description
username	str	required	Reddit username (without u/)
limit	int	25	Max items
sort	str	"new"	new, hot, top
activity_type	str	"all"	posts, comments, all

Returns: list[UserActivity]

Export Data

Save scraped data to JSON, CSV, or display as a formatted table.

JSON:

from reddipy.exporters import JSONExporter

posts = reddit.search("python", limit=20)
JSONExporter().export(posts, "results.json")

# Or print to stdout (no file path)
JSONExporter().export(posts)

CSV:

from reddipy.exporters import CSVExporter

posts = reddit.subreddit("python", sort="top", limit=50)
CSVExporter().export(posts, "posts.csv")

Console table (rich formatted):

from reddipy.exporters import ConsoleExporter

posts = reddit.search("python", limit=10)
ConsoleExporter().export(posts)

Exporting comments:

post, comments = reddit.post("https://reddit.com/r/python/comments/abc123/title/")

# Comments to JSON (includes post and comment data)
import json
combined = {
    "post": post.to_dict(),
    "comments": [c.to_dict() for c in comments]
}
with open("post_data.json", "w") as f:
    json.dump(combined, f, indent=2)

# Comments to CSV (flattened, with depth/parent_id columns)
CSVExporter().export(comments, "comments.csv")

Proxy Support (SDK)

Route requests through proxies to distribute load or bypass IP restrictions.

Single proxy:

reddit = Reddit(proxy="http://1.2.3.4:8080")

Multiple proxies with random rotation:

reddit = Reddit(proxy_file="proxies.txt")

Each request picks a random proxy from the file. Failed proxies are automatically skipped for 60 seconds.

proxies.txt format:

http://1.2.3.4:8080
http://5.6.7.8:3128
socks5://9.10.11.12:1080

SOCKS5 requires: pip install "reddipy[socks]"

Data Models

Every scraped item is a Python dataclass with typed fields. All models have a .to_dict() method for serialization.

Post

Field	Type	Description
id	str	Reddit post ID
title	str	Post title
author	str	Author username
subreddit	str	Subreddit name
score	int	Net upvotes
upvote_ratio	float	Upvote percentage (0.0 - 1.0)
num_comments	int	Total comment count
created_utc	datetime	Post creation time (UTC)
selftext	str	Post body text (empty for link posts)
url	str	Post URL or linked URL
permalink	str	Reddit permalink
flair	str or None	Post flair text

Comment

Field	Type	Description
id	str	Comment ID
author	str	Author username
body	str	Comment text
score	int	Net upvotes
created_utc	datetime	Comment creation time (UTC)
parent_id	str	Parent post/comment ID
depth	int	Nesting depth (0 = top-level)
is_submitter	bool	True if comment author is the post author
replies	list[Comment]	Nested reply comments

UserActivity

Field	Type	Description
type	str	"post" or "comment"
id	str	Item ID
author	str	Username
subreddit	str	Subreddit name
score	int	Net upvotes
created_utc	datetime	Creation time (UTC)
title	str or None	Post title (None for comments)
body	str or None	Comment body (None for posts)
permalink	str	Reddit permalink

Serialization:

post_dict = post.to_dict()
# {"id": "abc123", "title": "...", "created_utc": "2026-07-03T12:00:00+00:00", ...}

# datetime fields become ISO 8601 strings in to_dict()
# Nested comment replies are recursively serialized

Error Handling

Reddipy raises specific exceptions you can catch:

from reddipy.exceptions import (
    ReddipyError,           # base exception (catch-all)
    SubredditNotFoundError,  # subreddit is private, banned, or doesn't exist
    PostNotFoundError,       # post URL is invalid or deleted
    UserNotFoundError,       # user is suspended or doesn't exist
    RateLimitError,          # rate limited after 3 retries with backoff
    NetworkError,            # connection failure or timeout
    ProxyError,              # all proxies are dead
)

try:
    posts = reddit.subreddit("nonexistent_sub_12345")
except SubredditNotFoundError:
    print("Subreddit doesn't exist")
except RateLimitError:
    print("Rate limited, try again later or reduce --delay")
except NetworkError:
    print("Connection failed")

Advanced Configuration

reddit = Reddit(
    cookie="override_cookie",          # override stored cookie
    delay=1.0,                         # seconds between requests (default: 2.0)
    proxy="http://1.2.3.4:8080",       # single proxy
    proxy_file="proxies.txt",          # proxy list with random rotation
    user_agent="Custom Agent/1.0",     # custom User-Agent string
    verbose=True,                      # print progress to stderr
)

---

CLI Usage

All CLI commands use stored authentication by default. Run reddipy configure first.

Search (CLI)

# Basic search
reddipy search "python"

# With filters
reddipy search "machine learning" --limit 20 --sort top --time week

# Search within a subreddit
reddipy search "flask" --subreddit python --limit 10
reddipy search "hooks" -s react --sort new

Post (CLI)

# Scrape a post with all comments
reddipy post "https://www.reddit.com/r/python/comments/abc123/title/"

# Export to JSON
reddipy post "https://reddit.com/r/python/comments/abc123/title/" --export json -o post.json

Subreddit (CLI)

# Hot posts (default)
reddipy subreddit python

# Top posts this month
reddipy subreddit datascience --sort top --time month --limit 50

# New posts
reddipy subreddit technology --sort new --limit 20

# Rising
reddipy subreddit programming --sort rising

User (CLI)

# All activity
reddipy user spez --limit 10

# Posts only
reddipy user spez --type posts --limit 20

# Comments only, sorted by top
reddipy user spez --type comments --sort top

Export (CLI)

# JSON file
reddipy search "python" --export json -o results.json

# CSV file
reddipy subreddit python --export csv -o posts.csv

# Console table (default, no flag needed)
reddipy search "python" --limit 5

# Auto-generated filename (omit -o)
reddipy search "python" --export json
# Creates: reddipy_search_20260703_143000.json

Proxy Support (CLI)

# Single proxy
reddipy search "python" --proxy http://1.2.3.4:8080

# Multiple proxies with random rotation
reddipy search "python" --proxy-file proxies.txt

proxies.txt (one proxy per line):

http://1.2.3.4:8080
http://5.6.7.8:3128
socks5://9.10.11.12:1080

SOCKS5 requires: pip install "reddipy[socks]"

All CLI Flags

Common flags (available on all commands):

Flag	Description	Default
--limit	Max number of results	25
--sort	Sort order	varies by command
--time / -t	Time filter (day/week/month/year/all)	all
--export	Output format (json/csv/console)	console
--output / -o	Output file path	auto-generated
--cookie	Override stored reddit_session cookie	stored config
--cookie-file	Path to JSON file with cookies	-
--proxy	Single proxy URL	-
--proxy-file	Path to proxy list file	-
--delay	Seconds between requests	2.0
--user-agent	Custom User-Agent header	Chrome 149
--verbose / -v	Show progress and retry info	false

Command-specific flags:

Command	Flag	Description	Default
search	--subreddit / -s	Scope search to a subreddit	all of Reddit
user	--type	Filter activity (posts/comments/all)	all

Exit codes:

Code	Meaning
0	Success
1	General error (invalid args, unexpected failure)
2	Network error (connection failed, rate limited, proxies dead)
3	Not found (subreddit/post/user doesn't exist)

---

Rate Limiting

Reddipy has built-in protections to avoid getting blocked:

• Request delay: 2-second gap between requests (configurable with --delay or delay= parameter)
• Cooldown on 429: Automatic exponential backoff (2s, 4s, 8s) when rate-limited by Reddit
• Max retries: 3 attempts with increasing wait times before raising an error
• Dead proxy cooldown: Failed proxies are skipped for 60 seconds before being retried
• Realistic headers: All requests include 14 browser-standard headers (Accept, Sec-CH-UA, Sec-Fetch-*, etc.) matching a real Chrome browser fingerprint

Tips for staying safe:

• Keep delay at 2.0 or higher for normal use
• Use --verbose to see when retries and cooldowns happen
• For large scrapes (500+ items), use proxies to distribute load
• Don't run multiple instances on the same account simultaneously

Project Structure

reddipy/
    __init__.py       # Package exports: Reddit, Post, Comment, UserActivity
    sdk.py            # Python SDK (Reddit class)
    cli.py            # CLI entry point (argparse subcommands)
    client.py         # HTTP client (headers, rate limiting, pagination, proxies)
    config.py         # Persistent auth (~/.reddipy/config.json)
    models.py         # Post, Comment, UserActivity dataclasses
    exceptions.py     # ReddipyError, SubredditNotFoundError, etc.
    utils.py          # URL parsing, timestamp conversion, filename generation
    scrapers/
        base.py       # BaseScraper abstract class
        search.py     # SearchScraper
        post.py       # PostScraper
        subreddit.py  # SubredditScraper
        user.py       # UserScraper
    exporters/
        base.py       # BaseExporter abstract class
        json_exporter.py
        csv_exporter.py
        console.py    # Rich formatted tables

License

MIT