yp-dl 0.9.16

Asynchronous scraper to download youtube community posts

YoutubeCommunityScraper | yp-dl

yp-dl is an asynchronous scraper for downloading Youtube Community posts in json format.

Motivation

Youtube stops retrieving old community posts after 200 posts on a channel. There's no way to access/view older posts if you do not have the link to them or their ID.

Installation

pip install yp-dl

⚠️ Notice: For people on version 0.9.11 and before, you won't be able to use pip install upgrade yp-dl since I've upgraded the lxml version dependency above what is specified (refused to build on the old version suddenly for some reason). Please uninstall and then install yp-dl again to get the latest version.

Features

• Asynchronous support
• For every post it retrieves:
  • post_link
  • time_since
  • utc_timestamp at download
  • video_link
  • image_links
  • text_content
  • poll_content
• Update support for the json files when new posts are made
• Progress visualization during download

Usage

usage: yp-dl [-h] [-f FOLDER_PATH] [-r] [-u] [-v] [-o] [-d] link [link ...]

An asynchronous scraper that downloads youtube posts from youtube channels in json format.

positional arguments:
  link                  Provide any number of links. 
                        Link example: https://www.youtube.com/@3blue1brown

options:
  -h, --help            show this help message and exit
  -f FOLDER_PATH, --folder-path FOLDER_PATH
                        Provide the path of the folder you wish to store/update your json files. 
                        If it's in the current working directory (CWD), just type the folder 
                        name. If none is provided, everything will be stored/updated in the CWD.
  -r, --reverse         Reverses the order of the posts from oldest first to newest first. 
                        Be wary though, if you use this option with --update, your post order 
                        will be messed up.
  -u, --update          Appends the existing json file(s) with the new posts.
  -v, --verbose         Gives more details about what's going on when the program runs.
  -o, --overwrite-cookie
                        Overwrites the SOCS cookie in the cookies.txt file with a Default SOCS 
                        cookie within the project. Use if having problems retrieving posts.
  -d, --delete-cookie   Removes the cookie file to generate it again. Use if your SOCS key 
                        has expired (lifetime is 2 years).