mediaq

Hybrid CLI + REST API that searches media sites through their native search, sort and filter capabilities and resolves results into direct media URLs — optionally downloading them. Both halves share one core: the CLI runs in-process by default, can manage the server (mediaq server start|stop|status), and can drive a running server with --remote.

Stack: Python 3.12 · uv · Typer (CLI) · FastAPI/uvicorn (REST) · wreq (browser-emulated HTTP) · selectolax/Lexbor (HTML) · Pydantic v2 · SQLite (jobs & results) · YAML config.

Sites covered are adult sites. Respect their terms of use and rate limits; defaults are deliberately conservative (1 request/s, 2 concurrent per host).

Install

uv sync                      # development
uv run mediaq --help

uv tool install .            # or: install the `mediaq` command globally

Quick start

mediaq sites list                                   # capability matrix
mediaq sites test xvideos -q blonde                 # live smoke test of one adapter
mediaq --json sites list                            # global --json: JSON output everywhere
mediaq search "blonde" -s xvideos -s pornhub --sort newest --quality hd
mediaq search "blonde" --all --pages 2 -f jsonl -o results.jsonl --save
mediaq fetch --from results.jsonl                   # resolve direct media URLs
mediaq fetch "https://www.xvideos.com/video.abc/x" --download --dest ./dl --quality 720p
mediaq jobs list && mediaq jobs results <job-id> -f csv

Search options map onto each site's own parameters (--sort, --quality, --period, --orientation, --min-duration, --max-duration); anything a site cannot honour is dropped with a warning (--strict fails instead). Raw native parameters pass through with --extra key=value, and native query syntax (booru tags, nhentai filters, site: on NudeVista) goes straight into the query string.

fetch accepts page URLs, site:id / site:kind:id refs, --from a search output file, or --job a saved search. Aggregator results (PornMD, TubeGalore, iXXX, NudeVista) are resolved through the adapter of the site they point at.

Server

mediaq server start                 # detached uvicorn on 127.0.0.1:8765, pidfile + log file
mediaq server status | logs -f | restart | stop
mediaq server start --foreground    # run in the terminal

curl -s localhost:8765/health
curl -s -X POST localhost:8765/jobs/search -H 'content-type: application/json' \
     -d '{"query":"blonde","sites":["nhentai","rule34"],"pages":1}'
mediaq --remote http://127.0.0.1:8765 jobs list

Endpoints: GET /health, GET /sites[/{id}], POST /search (sync, ≤5 pages), POST /fetch, POST /jobs/search|fetch|download (async), GET /jobs[/{id}[/results|/downloads]], POST /jobs/{id}/cancel, DELETE /jobs/{id}, POST /jobs/purge, GET /downloads/{id}, POST /admin/shutdown (loopback-only; bearer token when server.admin_token is set). OpenAPI docs are served at /docs.

Configuration

~/.config/mediaq/config.yaml (or ./mediaq.yaml, --config PATH, MEDIAQ_CONFIG). Environment variables override the file: MEDIAQ_SERVER__PORT=9000. See config/mediaq.example.yaml for every key, and use mediaq config init|show|set to manage it.

Per-site settings live under sites.<id>: enabled, cookies (e.g. cf_clearance), user_agent, api_key/user_id (Rule34, Gelbooru, nhentai).

Sites

Generated by uv run python scripts/site_matrix.py.

Notes on what could be verified during development:

• Pornhub serves HLS renditions to guests; progressive MP4 (get_media) is usually empty. GIFs and album photos resolve to direct files.
• xHamster and HQPorner sign progressive URLs against the client IP; downloads fail with 403 Wrong key / 404 from networks whose egress IP differs per destination. HLS masters work. HLS-only sources are recorded but not downloaded in v1.
• XVideos/XNXX offer 240p/360p MP4 directly; higher qualities are HLS.
• Sex.com videos resolve to an HLS master plus a 240p MP4 preview (the page shows a regional age-verification overlay, but the media URLs are still exposed); GIFs are .webp animations.
• Interrupted downloads leave a .part file that the next attempt resumes with a Range request.
• Cloudflare challenges are detected and retried with backoff; a persistent challenge marks that site as failed for the request while other sites still return results.

Output formats

--format table|json|jsonl|csv, -o FILE. JSON Lines output is accepted back by fetch --from. SearchResult/FetchResult JSON includes per-site errors and warnings.

Development

scripts/check.sh                       # ruff format --check, ruff check, mypy, pytest
uv run pytest -m network               # opt-in live checks against every site
uv run python scripts/capture_fixtures.py [site ...]   # raw captures for parser drift
uv run python scripts/trim_fixture.py IN.html OUT.html 'css selector' ...

Adapters live in src/mediaq/sites/ (one module per site, shared mixins prefixed with _). Each adapter is a pure parser (parse_search, parse_detail) plus URL building, tested against trimmed HTML/JSON fixtures in tests/fixtures/<site>/. PLAN.md documents the research findings and the implementation log.

Hermes plugin

plugins/mediaq/ wraps this CLI as a Hermes plugin: mediaq_search, mediaq_fetch, mediaq_sites, mediaq_jobs and mediaq_config tools, three bundled skills, and a /mediaq slash command. It shells out to the installed mediaq binary, so Hermes' venv stays independent of this project's dependencies.

uv tool install .
hermes plugins install ./plugins/mediaq && hermes plugins enable mediaq

See plugins/mediaq/README.md.