grabberlib2 0.17.0

No project description provided

Grabber

Utilities to extract media from supported sources and publish to Telegram/Telegraph.

Quick Start

1. Install dependencies:

uv sync --dev

2. Configure environment in .env (at minimum):

• BOT_TOKEN
• TELEGRAPH_TOKEN
• SHORT_NAME
• AUTHOR_NAME
• AUTHOR_URL

3. Show supported entities:

uv run grabber -a

Run (CLI)

Example:

uv run grabber --publish --channel @your_channel --sources "https://example.com/post"

Send plain text to a channel:

uv run grabber --channel @your_channel --text "Some text"

Post text into a thread (discussion group):

uv run grabber --channel @your_channel --thread-id 123 --text "Some text in thread"

If --thread-id is omitted, grabber prints the resolved thread id so you can reuse it:

uv run grabber --channel @your_channel --text "Some text"

Output:

Text sent to @your_channel
Resolved thread id for reuse: --thread-id 123

Run Interactive Bot

uv run python -m grabber.core.bot.interactive_bot

Then use /post, select channel, and send source URLs.

Run Web UI

The web interface is served in two services:

• web-api: FastAPI backend (session-protected endpoints)
• web-ui: React dashboard

Setup

1. Add web environment variables:

WEB_UI_PASSWORD=your-strong-password
WEB_SESSION_SECRET=a-long-random-string
WEB_SESSION_TTL_SECONDS=86400

2. Start the stack:

docker compose up --build web-api web-ui

3. Open http://<your-vps-ip>:8888 (or map another public port in docker-compose.yml).

4. Log in with WEB_UI_PASSWORD. After login:

• Enter one or more URLs and target channel
• Optionally enable tag mode/video mode
• Send plain text by checking Post text only
• Optionally provide a thread id (--thread-id) for text replies/comments

Notes:

• The channel list is pulled from the existing bot CHANNELS mapping (src/grabber/core/settings.py).
• Entity list comes from QUERY_MAPPING.
• API routes are exposed under /api/* and proxied by the UI.

Pornhub Video-to-Channel

Single-video Pornhub URLs are supported for direct Telegram delivery.

Requirements

• A target channel is mandatory (--channel in CLI or bot channel picker).
• yt_dlp must be available in the runtime environment.
• PORNHUB_COOKIES_FILE_PATH must point to a valid Netscape-format cookies file.

Add to .env:

PORNHUB_COOKIES_FILE_PATH=/absolute/path/to/pornhub_cookies.txt

Supported Pornhub hosts

• pornhub.com
• www.pornhub.com
• m.pornhub.com

CLI example

uv run grabber \
  --publish \
  --channel -1003835153909 \
  --sources "https://www.pornhub.com/view_video.php?viewkey=ph5abc123"

Note: CLI --channel must be a Telegram username/ID (for example @channel_name or -100...), not the interactive bot display label.

Delivery behavior

• Chooses the best format expected to fit Telegram Bot API upload limits (<= 50MB).
• If the selected media is oversized, it sends a link message fallback to the channel.
• If --channel is missing, Pornhub execution fails fast with an explicit error.
• If PORNHUB_COOKIES_FILE_PATH is missing/invalid, execution fails fast.
• Checkpoint dedupe is applied, and successful sends are committed.

Troubleshooting

• PORNHUB_COOKIES_FILE_PATH is required...
  • Set PORNHUB_COOKIES_FILE_PATH in .env.
• ...does not exist or is not a file
  • Fix the path and permissions for the cookies file.
• yt_dlp is required to process Pornhub sources
  • Install dependencies in the same runtime where grabber executes.
• ERROR: [PornHub] ... Unable to extract title
  • The adapter now auto-falls back to HTML media parsing when yt_dlp metadata extraction fails.
  • If it still fails, refresh/export cookies again and update yt-dlp in your runtime.
• Oversized media warnings
  • Expected behavior; the source falls back to link delivery.