vidurl 0.3.0

Extract video URLs from web pages and generate curl download commands

vidurl

A more powerful, cleverer yt-dlp. Given any web page — a single-video page, a thumbnail-grid listing, or something obscure yt-dlp doesn't know about — vidurl downloads the videos.

How it works

Three-tier escalation per URL, cheapest first:

1. yt-dlp handles whatever it can natively (thousands of sites, including playlists/channels).
2. Playwright loads the page and sniffs the rendered DOM, JS, network responses, and embedded iframes for a video URL — then downloads with curl. For listing pages, vidurl scrapes the DOM for video-page links and recurses.
3. LLM (optional, opt-in) — when the heuristics fail, vidurl can ask a large language model (via scrapegraphai) to identify the video URL or video-page links on the rendered DOM.

Installation

uv sync
uv run playwright install chromium

yt-dlp and playwright are installed automatically. The LLM tier is optional:

uv sync --extra llm           # pulls in scrapegraphai

Also requires curl in PATH.

Usage

# yt-dlp does the work
vidurl https://www.youtube.com/watch?v=dQw4w9WgXcQ

# Site yt-dlp doesn't know — Playwright finds the video
vidurl https://example.com/embedded-video-page

# Listing of videos — vidurl visits each link
vidurl https://example.com/gallery
vidurl https://example.com/gallery --listing --link-selector "a.video-card"

# Don't download, just print the commands
vidurl https://example.com/page --dry-run

# Enable LLM fallback
vidurl https://example.com/weird-page \
    --llm-provider anthropic --llm-model claude-haiku-4-5

Flags

Flag	Purpose
--output-dir / -o DIR	Where to save files (default: cwd)
--dry-run	Print the yt-dlp / curl commands instead of executing
--no-ytdlp	Skip the yt-dlp tier; go straight to Playwright
--ytdlp-args 'STR'	Extra args appended to the yt-dlp invocation
--listing	Force listing mode (skip per-page video extraction)
--no-listing	Never recurse into links
--link-selector CSS	Use this CSS selector to find video-page links
--link-pattern REGEX	Only follow links whose absolute URL matches
--min-links N	Minimum links for listing auto-detect (default 3)
--max-pages N	Max listing pages to walk (default 10)
--no-paginate	Disable pagination — process only the first listing page
--next-selector CSS	CSS selector for the next-page link
--next-pattern REGEX	Regex; treat as next-page link only if URL matches
--page-url-template URL	URL template with {n}; vidurl walks 2..max-pages
--llm-provider P	anthropic, openai, groq, google, ollama, ...
--llm-model M	Model id
--no-llm	Disable the LLM tier even if provider/model are set
--no-headless	Show the browser window
--timeout S	Page load timeout (default 15s)
--verbose / --quiet	Logging

LLM tier

The LLM tier is off by default. To enable, pass both --llm-provider and --llm-model (or set them in config.json). API keys are read from environment first, then from gnome-keyring via secret-tool under service=env, key=<PROVIDER_KEY>. If a key is present but provider/model are not set, vidurl logs a hint and stays off — no silent spend.

Pagination

When a listing page is detected, vidurl walks subsequent pages automatically (capped by --max-pages, default 10). Next-page discovery tries, in order:

1. <link rel="next"> or <a rel="next">.
2. Anchor text or aria-label matching Next, ›, →, », More (filtered to avoid "next video" navigation).
3. URL-template inference (?page=N, /page/N/, &offset=N, trailing /N), validated with a HEAD/GET probe.
4. LLM fallback (if the LLM tier is enabled).

Override with --next-selector, --next-pattern, or --page-url-template URL (e.g. https://example.com/list?page={n}). Disable with --no-paginate.

What's intentionally not (yet) supported

• "Load more" buttons and infinite scroll — these need click-and-wait logic without a URL change.
• Parallel downloads of multiple listing-page videos — sequential for now.

Configuration

Pass a JSON file via --config path/to/config.json. See config.example.json for all fields.

Development

uv sync --extra dev
uv run pytest        # no tests yet

License

AGPL-3.0-or-later. See LICENSE.