videosrc

Web crawler to extract video data from HTML, MRSS and more.

This library primarily extracts the source attributes / tags from the HTML5 <video/> element.

Each crawler accepts an opaque state value which allows it to resume a crawl at a later time. In addition, each crawler accepts an argument save_state which should be a callback that receives this opaque state value. A caller’s job is to store that state for future use.

Motivation

I needed to be able to enumerate videos from multiple different websites and formats.

Backends

A given URL is handled by a specific backend which knows how to interpret the HTML.

HTML

Meant to read from a static HTML page or index page generated by a webserver when viewing a directory full of video files. Looks for videos in <video/> and <a/> tags.

MRSS

Media RSS is a syndication format for media. As such we don’t really need to crawl the given URL but simply parse it and return the listed videos.

Peertube

Utilizes the PeerTube REST API to fetch video information.

Odysee

Utilizes the Odysee REST API to fetch video information.

Rumble

Uses web scraping to fetch video information. Utilizes Chromium to allow Javascript execution.

Timcast

Uses pyppeteer to login and scrape video urls, then fetches video data from Rumble.

Twitter

Uses snscrape to scrape media from a user’s tweets.

Metadata

Release files for videosrc 0.1.10

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distribution (wheel)

Table of built distributions (wheels) for videosrc 0.1.10
File	Interpreter	ABI	Platform	Reset
videosrc-0.1.10-py3-none-any.whl 92.2 kB May 21, 2023	Python 3	none	any	Details

Release files / videosrc-0.1.10-py3-none-any.whl

Download URL	videosrc-0.1.10-py3-none-any.whl
Size	92.2 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`28cf42f5f17107746a42ee402277699d9349c2968836f0539ce923a986681025`
BLAKE2b-256 checksum How to use checksums	`1c23cc147dbd3f77e916e276d6eb564e5ae48ee94ff19e821f462b05486270ce`
Upload date	May 21, 2023
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/3.8.0 colorama/0.4.4 importlib-metadata/4.6.4 keyring/23.5.0 pkginfo/1.8.2 readme-renderer/34.0 requests-toolbelt/0.9.1 requests/2.25.1 rfc3986/1.5.0 tqdm/4.57.0 urllib3/1.26.5 CPython/3.10.6`