twarc-videos
This twarc plugin uses youtube_dl to download videos and their metadata from tweets. This is nice because youtube_dl downloads video from many more platforms than YouTube including Twitter itself.
To use twarc-videos first you need to install it:
pip install twarc-videos
Now you can collect data using the core twarc utility. For example this search finds tweets that mention the word "nirvana" and also have native video (Twitter video) or a link to YouTube:
twarc2 search 'nirvana (has:videos OR url:"https://youtu.be")' > nirvana-tweets.jsonl
And you have a new subcommand videos that is supplied by twarc-videos.
twarc2 videos nirvana-tweets.jsonl
Once it is finished you will have a new videos directory that looks something
like:
videos
├── archive.txt
├── mapping.tsv
├── twitter
│ ├── 1339223561731530753
│ │ ├── Psychedelia_-_Nirvana_-_Come_As_You_Are.description
│ │ ├── Psychedelia_-_Nirvana_-_Come_As_You_Are.info.json
│ │ └── Psychedelia_-_Nirvana_-_Come_As_You_Are.mp4
│ ├── 1341668409428353025
│ │ ├── Rt_Your_Fav_Bands_-_Nirvana_Come_As_You_Are.description
│ │ ├── Rt_Your_Fav_Bands_-_Nirvana_Come_As_You_Are.info.json
│ │ └── Rt_Your_Fav_Bands_-_Nirvana_Come_As_You_Are.mp4
│ ├── 1374212180002926594
│ │ ├── Hanna_-_She_s_in_Nirvana....description
│ │ ├── Hanna_-_She_s_in_Nirvana....info.json
│ │ └── Hanna_-_She_s_in_Nirvana....mp4
│ ├── 1374467789885378569
│ │ ├── MUSIC_NOSTALGIA_-_Nirvana_The_Man_Who_Sold_The_World_..description
│ │ ├── MUSIC_NOSTALGIA_-_Nirvana_The_Man_Who_Sold_The_World_..info.json
│ │ └── MUSIC_NOSTALGIA_-_Nirvana_The_Man_Who_Sold_The_World_..mp4
│ ├── 1374469206226264067
│ │ ├── Take_it_easy_-_Abuelo_donde_andas_Nirvana.description
│ │ ├── Take_it_easy_-_Abuelo_donde_andas_Nirvana.info.json
│ │ └── Take_it_easy_-_Abuelo_donde_andas_Nirvana.mp4
│ ├── 1374631023502360576
│ │ ├── OraEtLabora_-_Reel_Stories_-_Dave_Grohl_is_on_@bbctwo_this_Saturday_at_10.30pm...talking_@Nirvana_amp_@foofighters_with_Dermot_@radioleary_@wearecraftuk.description
│ │ ├── OraEtLabora_-_Reel_Stories_-_Dave_Grohl_is_on_@bbctwo_this_Saturday_at_10.30pm...talking_@Nirvana_amp_@foofighters_with_Dermot_@radioleary_@wearecraftuk.info.json
│ │ └── OraEtLabora_-_Reel_Stories_-_Dave_Grohl_is_on_@bbctwo_this_Saturday_at_10.30pm...talking_@Nirvana_amp_@foofighters_with_Dermot_@radioleary_@wearecraftuk.mp4
│ ├── 1374656171844329477
│ ├── 1374656880694292483
│ ├── 1374660019241762817
│ ├── 1374664809078272000
│ └── 1374671562016661506
│ ├── John_-_Nirvana_-_In_Bloom_Live_at_Reading_1992_@YouTube.description
│ ├── John_-_Nirvana_-_In_Bloom_Live_at_Reading_1992_@YouTube.info.json
│ └── John_-_Nirvana_-_In_Bloom_Live_at_Reading_1992_@YouTube.mp4
└── youtube
├── 5X9CGFQyjN4
│ ├── Heart-Shaped_Box_Nirvana_Music_Box.description
│ ├── Heart-Shaped_Box_Nirvana_Music_Box.en.vtt
│ ├── Heart-Shaped_Box_Nirvana_Music_Box.info.json
│ └── Heart-Shaped_Box_Nirvana_Music_Box.mp4
├── AhcttcXcRYY
│ ├── Nirvana_-_About_A_Girl_MTV_Unplugged.description
│ ├── Nirvana_-_About_A_Girl_MTV_Unplugged.en.vtt
│ ├── Nirvana_-_About_A_Girl_MTV_Unplugged.info.json
│ └── Nirvana_-_About_A_Girl_MTV_Unplugged.mp4
├── AXU-LaaO_xQ
│ ├── Nirvana_Drain_You_lyrics_sub_espanol.description
│ ├── Nirvana_Drain_You_lyrics_sub_espanol.info.json
│ └── Nirvana_Drain_You_lyrics_sub_espanol.mp4
├── D742dNm1f8Q
│ ├── Nirvana_-_In_Bloom_Live_at_Reading_1992.description
│ ├── Nirvana_-_In_Bloom_Live_at_Reading_1992.info.json
│ └── Nirvana_-_In_Bloom_Live_at_Reading_1992.mp4
├── -fh-bqSV73E
│ ├── Becoming_a_minimalist_w_Matt_D_Avella.description
│ ├── Becoming_a_minimalist_w_Matt_D_Avella.en.vtt
│ ├── Becoming_a_minimalist_w_Matt_D_Avella.info.json
│ └── Becoming_a_minimalist_w_Matt_D_Avella.mp4
├── hTWKbfoikeg
│ ├── Nirvana_-_Smells_Like_Teen_Spirit_Official_Music_Video.description
│ ├── Nirvana_-_Smells_Like_Teen_Spirit_Official_Music_Video.en.vtt
│ ├── Nirvana_-_Smells_Like_Teen_Spirit_Official_Music_Video.info.json
│ └── Nirvana_-_Smells_Like_Teen_Spirit_Official_Music_Video.mp4
├── jWkSt4G8F18
│ ├── Nirvana_healing_centre_overview.description
│ ├── Nirvana_healing_centre_overview.info.json
│ └── Nirvana_healing_centre_overview.mp4
├── MW6E_TNgCsY
│ ├── Everclear_-_Santa_Monica_Official_Music_Video.description
│ ├── Everclear_-_Santa_Monica_Official_Music_Video.info.json
│ └── Everclear_-_Santa_Monica_Official_Music_Video.mp4
├── n6P0SitRwy8
│ ├── Nirvana_-_Heart-Shaped_Box.description
│ ├── Nirvana_-_Heart-Shaped_Box.info.json
│ └── Nirvana_-_Heart-Shaped_Box.mp4
├── OgeR2oqZGTs
│ ├── Nirvana_-_The_Man_Who_Sold_The_World_Live_On_MTV_Unplugged_1993_Unedited.description
│ ├── Nirvana_-_The_Man_Who_Sold_The_World_Live_On_MTV_Unplugged_1993_Unedited.en.vtt
│ ├── Nirvana_-_The_Man_Who_Sold_The_World_Live_On_MTV_Unplugged_1993_Unedited.info.json
│ └── Nirvana_-_The_Man_Who_Sold_The_World_Live_On_MTV_Unplugged_1993_Unedited.mp4
├── v9RY25eImcw
│ ├── Nirvana_-_Smells_Like_Teen_Spirit_Cover_RADIO_TAPOK.description
│ ├── Nirvana_-_Smells_Like_Teen_Spirit_Cover_RADIO_TAPOK.en.vtt
│ ├── Nirvana_-_Smells_Like_Teen_Spirit_Cover_RADIO_TAPOK.info.json
│ └── Nirvana_-_Smells_Like_Teen_Spirit_Cover_RADIO_TAPOK.mp4
├── ycHvL3W3_PA
│ ├── Nirvana_-_Where_Did_You_Sleep_Last_Night_8D_Audio.description
│ ├── Nirvana_-_Where_Did_You_Sleep_Last_Night_8D_Audio.info.json
│ └── Nirvana_-_Where_Did_You_Sleep_Last_Night_8D_Audio.mp4
└── y-lQgqHD8Xs
├── dodo_tofubeats_-_nirvana_Official_Music_Video.description
├── dodo_tofubeats_-_nirvana_Official_Music_Video.info.json
└── dodo_tofubeats_-_nirvana_Official_Music_Video.mp4
The video/mapping.tsv file is a tab separated value file of video URLs found
and their corresponding location in disk.
Testing
To run the tests you will need create a .env file that looks like:
BEARER_TOKEN=YOUR_TOKEN_HERE
And then:
python setup.py test
Metadata
Release files for twarc-videos 0.0.6
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| twarc-videos-0.0.6.tar.gz | 5.6 kB | Without provenance | Details for twarc-videos-0.0.6.tar.gz |
Release files / twarc-videos-0.0.6.tar.gz
| Download URL | twarc-videos-0.0.6.tar.gz |
|---|---|
| Size | 5.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
7790d77888f25bb4262b6761f40774c8caf72fc2f713f6c3a44936e377bac2d3
|
|
BLAKE2b-256 checksum How to use checksums |
484d6b463a3b1ce978bf33fcdde66dab4bbdc07ba5a0a717927013a7b9129e07
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.2.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/45.2.0 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.8.5
|
Security for twarc-videos 0.0.6
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.