bili-downloader-from-scratch 0.1.0

A from-scratch Bilibili downloader learning project.

bili-downloader-from-scratch

A from-scratch learning project for building a Bilibili downloader step by step.

This repository is intentionally scoped as an MVP path, not a clone of a full desktop app. The goal is to grow the downloader in layers: parsing first, stream selection second, downloading third, merging fourth, and GUI last.

Current status

Implemented:

• Runnable CLI entrypoint
• inspect command with real metadata parsing for normal Bilibili video URLs and Bangumi episode/season URLs
• streams command with real DASH stream manifest parsing for normal videos and Bangumi episodes
• plan command that writes metadata + stream based JSON plans
• download command that downloads media files, with optional subtitle and danmaku sidecars
• batch-download command for multi-link downloads, with configurable --jobs concurrency, list-source expansion, and optional multi-page expansion
• history list / history show for querying saved records and mutable run state
• resume command for restarting failed or partial plan / download / batch-download runs
• pause command for pausing active runs, queued runs, or one batch item by index
• retry-failed command for retrying failed items in a saved batch run
• cancel command for requesting cancellation of active or queued runs
• app_service facade for GUI/CLI task creation, listing, detail lookup, pause, cancellation, retry, and resume
• auth-check command for validating whether configured Bilibili cookies are loaded and logged in
• Lightweight tkinter GUI MVP with task list, new download, task detail, Cookie, and settings pages
• Support for BV..., av..., Bangumi ss... / ep... / md..., full video URLs, and b23.tv short URLs
• Batch expansion for favorites, uploader collections, Bangumi seasons, and UP space video lists
• Optional subtitle sidecar download in srt or raw json format
• Optional danmaku XML sidecar download
• WBI signing flow for the live video detail API
• Best-available quality selection, with cookie-enabled higher tiers when the account can access them
• Premium audio parsing for Dolby and Hi-Res lossless tracks
• Built-in Windows ffmpeg runtime for merging, with config/PATH overrides and automatic fallback to separate audio/video files when ffmpeg is unavailable
• CLI progress output during downloads
• --overwrite support for repeated downloads
• Cookie file loading for authenticated requests
• Immutable per-run JSON records saved under the local history_dir, including failures
• Mutable run snapshots saved under the local runs_dir for task resume and future GUI polling
• Batch item phase/progress snapshots in run.json for active GUI polling
• Request-style pause checkpoints for active download and batch runs
• Request-style cancellation checkpoints for active download and batch runs
• Structured error categories for auth, network, risk, quality, ffmpeg, io, parser, cancelled, paused, and unknown
• Retry/backoff for retryable metadata, manifest, and media download failures
• Range-based resume from retained .part files
• Config loading and log initialization
• pytest, ruff, and GitHub Actions CI configuration
• Unit and integration-style tests for parser, stream, downloader, task, and real-scenario flows

GUI MVP:

• Task list with status, phase, progress, speed, ETA, cancel, resume, and open-folder actions
• New download page for single URL or multi-line batch downloads
• Task detail page with raw run.json and batch item rows
• Cookie configuration page with save and login-check actions
• Settings page for output directory, default concurrency, and ffmpeg path

Support scope

Current backend coverage is no longer limited to plain single-video downloads:

• normal videos, including optional multi-page expansion via batch-download --expand-pages
• Bangumi episode URLs and season/media URLs
• favorites, uploader collections, and UP space video lists as batch sources
• optional subtitle and danmaku sidecar downloads during plan / download / batch-download

Still not positioned as a full product-grade desktop downloader for every Bilibili surface. Live streams, paid courses, broader GUI controls for the new sidecar/list features, and deeper real-device compatibility work remain future work.

Structure

bili-downloader-from-scratch/
|-- main.py
|-- pyproject.toml
|-- requirements.txt
|-- LICENSE
|-- THIRD_PARTY_NOTICES.md
|-- docs/
|-- src/
|   `-- bili_downloader/
|       |-- __init__.py
|       |-- __main__.py
|       |-- app_service.py
|       |-- auth.py
|       |-- cli.py
|       |-- config.py
|       |-- downloader.py
|       |-- errors.py
|       |-- logging_setup.py
|       |-- models.py
|       |-- network.py
|       |-- parser.py
|       |-- records.py
|       |-- services/
|       |   |-- __init__.py
|       |   |-- history.py
|       |   `-- tasks.py
|       `-- streams.py
|-- tools/
|   `-- ffmpeg-8.1-essentials_build/
|       |-- LICENSE
|       |-- README.txt
|       `-- bin/
|           `-- ffmpeg.exe
`-- tests/
    |-- test_app_service.py
    |-- test_auth.py
    |-- test_cli.py
    |-- test_downloader.py
    |-- test_errors.py
    |-- test_history.py
    |-- test_network.py
    |-- test_parser.py
    |-- test_records.py
    |-- test_streams.py
    `-- test_tasks.py

Run

cd D:\pycode\bili-downloader-from-scratch
python -m pip install -e ".[dev]"
bili-downloader --help
python .\main.py --help
python .\main.py inspect "https://www.bilibili.com/video/BV1xx411c7mD"
python .\main.py inspect "https://www.bilibili.com/bangumi/play/ss1129"
python .\main.py streams "BV1xx411c7mD"
python .\main.py plan "BV1xx411c7mD" --with-subtitles --with-danmaku
python .\main.py download "BV1xx411c7mD" --audio-only --overwrite
python .\main.py download "https://www.bilibili.com/bangumi/play/ep2001" --with-subtitles --with-danmaku
python .\main.py batch-download BV1xx411c7mD av170001 --audio-only --overwrite
python .\main.py batch-download BV1xx411c7mD av170001 --jobs 2 --overwrite
python .\main.py batch-download BV1xx411c7mD --expand-pages
python .\main.py batch-download "https://www.bilibili.com/list/ml123456"
python .\main.py batch-download "https://space.bilibili.com/2/video" --with-subtitles
python .\main.py history list --limit 10
python .\main.py history show <record-id-or-run-id>
python .\main.py resume <run-id> --jobs 2
python .\main.py pause <run-id> --reason "pause from GUI"
python .\main.py pause <run-id> --item 1 --reason "hold one item"
python .\main.py retry-failed <run-id> --jobs 2
python .\main.py cancel <run-id> --reason "stop from GUI"
python .\main.py gui
python .\main.py config
python .\main.py auth-check
python .\main.py --cookie-file .\cookies.txt streams "BV1xx411c7mD"
python -m ruff check .
python -m pytest

The bili-downloader console script is available after installing the project. python .\main.py ... remains useful when running directly from a source checkout.

FFmpeg policy

Releases include a bundled Windows ffmpeg.exe under tools/ffmpeg-8.1-essentials_build/bin. The downloader resolves ffmpeg in this order:

1. BILI_DOWNLOADER_FFMPEG, set from the config file or GUI settings page
2. the bundled ffmpeg.exe from an installed package or source checkout
3. ffmpeg found on PATH

If no usable ffmpeg is found, downloads still succeed by saving separate video and audio files. The release does not auto-download ffmpeg on first start.

License

Project source code is MIT licensed. Release artifacts also include a bundled Windows FFmpeg build under GPL v3; see THIRD_PARTY_NOTICES.md and tools/ffmpeg-8.1-essentials_build/LICENSE.

Cookie files can be provided as:

• raw header text such as SESSDATA=...; bili_jct=...
• JSON object such as {"SESSDATA": "...", "bili_jct": "..."}
• JSON array with name / value entries
• Netscape cookies.txt

By default the CLI looks for cookies at the cookie_file path printed by:

python .\main.py config

Create that file with your own Bilibili browser cookies, then verify login state with:

python .\main.py auth-check

For a one-off cookie file, use:

python .\main.py --cookie-file .\cookies.txt auth-check
python .\main.py --cookie-file .\cookies.txt download "BV1xx411c7mD" --quality 1080p --overwrite

auth-check only prints cookie names and account status. It intentionally does not print cookie values.

Each plan, download, and batch-download run saves two local artifacts:

• a final JSON record in history_dir, with status, exit_code, request context, result payload, and any error details
• a mutable run.json snapshot in runs_dir, with run_id, phase, attempts, item-level batch state, active progress, and links to emitted history records

This split keeps immutable audit records and resumable task state separate, which is a better base for GUI task lists and interrupted-run recovery.

Errors are serialized with a stable category, retryable flag, and stage. Retryable metadata and manifest failures are retried before the command fails. Media track downloads retry each URL with exponential backoff and still fall through to backup URLs when available. Bilibili HTTP 412 and request-frequency failures are reported as risk, while expired or missing login cookies are reported as auth.

Batch concurrency is capped at 3 jobs, and all network opens pass through a small per-host rate limiter to reduce request bursts. Media downloads preflight known Content-Length values against available disk space, reject illegal Windows path components, keep .part files for resumable failures, and remove completed merge work directories after successful ffmpeg output.

Batch resume skips successful items, reruns failed or pending items, and resumes paused items. resume --jobs N can override the stored concurrency for a resumed batch. retry-failed <run-id> only requeues failed batch items and leaves successful or paused items untouched.

Pause is request-based. pause <run-id> writes pause_requested and moves active runs to pause_requested until the running process reaches a checkpoint. Active runs check this flag during phase transitions, progress updates, and media chunk downloads, then finalize as paused while keeping .part files for Range resume. pause <run-id> --item N pauses one batch item without pausing the rest of the batch.

Cancellation is request-based. cancel <run-id> writes cancel_requested into the run state. Active runs check this flag during phase transitions, progress updates, and media chunk downloads, then finalize as cancelled with exit code 130.

GUI and other application clients should use bili_downloader.app_service instead of depending on CLI argument parsing. The facade exposes create_download_task, create_batch_download_task, list_tasks, get_task_detail, pause_task, cancel_task, retry_failed_task, and resume_task; the CLI now uses the same facade.

The GUI saves its settings to the default app config file under app_home. Use the settings page for output_dir, default batch concurrency, and an optional ffmpeg executable path to override the bundled runtime. The newly added subtitle / danmaku / list-expansion controls are currently exposed through the CLI and app_service API first.

Next step

1. Expose subtitle, danmaku, and list-expansion options in the GUI.
2. Expand real-device regression coverage across more account types, regions, member-only videos, and larger list sources.
3. Package the GUI as a standalone executable.