Skip to main content

Livechat scraper for Youtube video streams

Project description

youtube-livechat-scraper

A tool to scrape youtube livechat data I came up with while watching vtubers. It rips almost all data from a VOD's livechat including the following:

  • Chat messages
  • Superchats
  • Memberships joined
  • Memberships gifted and received

All of the data can be wrapped up in a large raw JSON object that also contains lots of metadata from the responses, like the author, the time the message got sent, etc.

USAGE:

Requires the following python packages:

  • Python3

  • BeautifulSoup

  • Requests

  • Import the LiveChatScraper from scrapers.liveChatScraper to wherever you want to make the scraping call.

  • Find a VOD URL and copy it

  • Create a LiveChatScraper object and pass in the VOD's URL.

  • Call the scrape() method on the created scraper object and the scrape will run.

  • Once the scrape is completed, you can call ouputMessages() to get a dictionary with all the scraped data.

  • You can all save the scraped data as a JSON to a fill by calling the writeToFile method passing the OUTPUT_JSON constant

  • example.py has a working example which saves the data to different formats

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

youtube_livechat_scraper_ohn0-1.0.2.tar.gz (13.9 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

youtube_livechat_scraper_ohn0-1.0.2-py3-none-any.whl (23.9 kB view details)

Uploaded Python 3

File details

Details for the file youtube_livechat_scraper_ohn0-1.0.2.tar.gz.

File metadata

File hashes

Hashes for youtube_livechat_scraper_ohn0-1.0.2.tar.gz
Algorithm Hash digest
SHA256 0bc7ad283ad4c04dce4a213ff49dc59b5956023632803a8b32b8e6c53fccab1d
MD5 a265f9e21b62e3b4aca4177bbdb4abc7
BLAKE2b-256 864c47421cc275e9ea8fade8150bdfb6566f2e34d09e4966caca81a68e777cbb

See more details on using hashes here.

File details

Details for the file youtube_livechat_scraper_ohn0-1.0.2-py3-none-any.whl.

File metadata

File hashes

Hashes for youtube_livechat_scraper_ohn0-1.0.2-py3-none-any.whl
Algorithm Hash digest
SHA256 330bbf501624ef9e92444781bb47a1425c8dc824ad7b15d7b0f6a6ddc6e9af7b
MD5 5fb6cfdfddd2c5f33f423d27ec72d33a
BLAKE2b-256 d25500d859d388967c2b48f995b51443f3f9017eeb0356ec27f5acb62fbc758c

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page