xklb 1.14.3

xk library

lb: xk media library

A wise philosopher once told me, "The future is autotainment".

Manage large media libraries. Think Plex but minimalist and CLI-only. Primary usage is local filesystem but also supports some virtual constructs like tracking video playlists (eg. YouTube subscriptions) and browser tabs.

Requires ffmpeg

Install

pip install xklb

Quick Start -- filesystem

1. Extract Metadata

For thirty terabytes of video the initial scan takes about four hours to complete. After that, rescans of the same path (or any subpaths) are much quicker--only new files will be read by ffprobe.

lb extract tv.db ./video/folder/

2. Watch / Listen from local files

wt tv.db                          # the default post-action is to do nothing after playing
wt tv.db --post-action delete     # delete file after playing
lt finalists.db --post-action=ask # ask to delete after playing

Quick Start -- tube (track online video playlists)

1. Download Metadata

Download playlist and channel metadata. Break free of the YouTube algo~

lb tubeadd educational.db https://www.youtube.com/c/BranchEducation/videos

You can add more than one at a time.

lb tubeadd maker.db https://www.youtube.com/c/CuriousMarc/videos https://www.youtube.com/c/element14presents/videos/ https://www.youtube.com/c/DIYPerks/videos

And you can always add more later--even from different websites.

lb tubeadd maker.db https://vimeo.com/terburg

To prevent mistakes the default configuration is to download metadata for only the newest 20,000 videos per playlist/channel.

lb tubeadd maker.db --yt-dlp-config playlistend=1000

Be aware that there are some YouTube Channels which have many items--for example the TEDx channel has about 180,000 videos. Some channels even have upwards of two million videos. More than you could likely watch in one sitting. On a high-speed connection (>500 Mbps), it can take up to five hours just to download the metadata for 180,000 videos. My advice: start with the 20,000.

1a. Get new videos for saved playlists

Tubeupdate will go through all added playlists and fetch metadata of any new videos not previously seen.

lb tubeupdate

2. Watch / Listen from websites

lb tubewatch maker.db

If you like this I also have a web version--but this Python version has more features and it can handle a lot more data.

Quick Start -- tabs (visit websites on a schedule)

tabs provides a way to organize your visits to URLs that you want to visit every once in a while.

If you want to track changes to websites over time there are better tools out there, like huginn, urlwatch, or changedetection.io.

The use-case of tabs are websites that you know are going to change: subreddits, games, or tools that you want to use for a few minutes every certain frequency (eg. daily/weekly/monthly/quarterly/yearly).

1. Add your websites

lb tabsadd --frequency monthly --category fun https://old.reddit.com/r/Showerthoughts/top/?sort=top&t=month https://old.reddit.com/r/RedditDayOf/top/?sort=top&t=month

Depending on your shell you may need to quote the URLs.

If you use Fish shell know that you can enable features to make pasting easier:

set -U fish_features stderr-nocaret qmark-noglob regex-easyesc ampersand-nobg-in-token

Also I recommend turning Ctrl+Backspace into a super-backspace for repeating similar commands with different long URLs:

echo 'bind \b backward-kill-bigword' >> ~/.config/fish/config.fish

If you have an existing line-deliminated file you can import it like this:

lb tabsadd -f monthly -c reddit ~/lb/tabs.db (cat ~/mc/reddit.monthly.cron)

2. Add lb tabs to cron

lb tabs is meant to run once per day. Here is how you would configure it with crontab:

45 9 * * * DISPLAY=:0 lb tabs /home/my/tabs.db

If things aren't working you can use at to simulate a similar environment as cron

echo 'DISPLAY=:0 lb tabs /full/path/to/tabs.db' | at NOW

You can also invoke tabs manually:

lb tabs -L 1  # open one tab

Things to know

When the database file path is not specified, video.db will be created / used.

lb extract ./tv/

The same for audio: audio.db will be created / used.

lb extract --audio ./music/

Likewise, fs.db from:

lb extract --filesystem /any/path/

If you want to specify more than one directory you need to mention the db file explicitly.

lb extract --filesystem one/
lb extract --filesystem fs.db one/ two/

Organize via separate databases.

lb extract --audio both.db ./audiobooks/ ./podcasts/
lb extract --audio audiobooks.db ./audiobooks/
lb extract --audio podcasts.db ./podcasts/ ./another/more/secret/podcasts_folder/

Usage

Repeat

lt                  # listen to 120 random songs (DEFAULT_PLAY_QUEUE)
lt --limit 5        # listen to FIVE songs
lt -l inf -u random # listen to random songs indefinitely
lt -s infinite      # listen to songs from the band infinite

Watch longest videos

wt tv.db --sort duration desc

Watch specific video series in order

wt tv.db --search 'title of series' --play-in-order

There are multiple strictness levels of --play-in-order. If things aren't playing in order try adding more Os

wt tv.db --search 'title of series' -O    # default
wt tv.db --search 'title of series' -OO   # slower, more complex algorithm
wt tv.db --search 'title of series' -OOO  # most strict

See how many corrupt videos you have

lb wt -w 'duration is null' -p a

Listen to OSTs on chromecast groups

lt -cast -cast-to 'Office pair' -s '  ost'

Exercise and watch TV that doesn't have subtitles

wt -u priority -w subtitle_count=0

Print a list of files below a 1280px resolution

wt -w 'width<1280' -p f

Play files under a certain throughput

Bitrate information is not explicitly saved but you can use file size and duration as a proxy:

wt -w 'size/duration<50000'

Check if you've downloaded something before

wt -u duration --print -s 'video title'

View how much time you have listened to music

lb lt -w play_count'>'0 -p a

See how much video you have

lb wt video.db -p a
╒═══════════╤═════════╤═════════╤═════════╕
│ path      │   hours │ size    │   count │
╞═══════════╪═════════╪═════════╪═════════╡
│ Aggregate │  145769 │ 37.6 TB │  439939 │
╘═══════════╧═════════╧═════════╧═════════╛
Total duration: 16 years, 7 months, 19 days, 17 hours and 25 minutes

Search the filesystem

You can also use lb for any files:

$ lb extract -fs ~/d/41_8bit/

$ lb fs fs.db -p a -s mario luigi
╒═══════════╤══════════════╤══════════╤═════════╕
│ path      │   sparseness │ size     │   count │
╞═══════════╪══════════════╪══════════╪═════════╡
│ Aggregate │            1 │ 215.0 MB │       7 │
╘═══════════╧══════════════╧══════════╧═════════╛

$ lb fs -p -s mario -s luigi -s jpg -w is_dir=0 -u 'size desc'
╒═══════════════════════════════════════╤══════════════╤═════════╕
│ path                                  │   sparseness │ size    │
╞═══════════════════════════════════════╪══════════════╪═════════╡
│ /mnt/d/41_8bit/roms/gba/media/images/ │      1.05632 │ 58.2 kB │
│ Mario & Luigi - Superstar Saga (USA,  │              │         │
│ Australia).jpg                        │              │         │
├───────────────────────────────────────┼──────────────┼─────────┤
│ /mnt/d/41_8bit/roms/gba/media/box3d/M │      1.01583 │ 44.4 kB │
│ ario & Luigi - Superstar Saga (USA,   │              │         │
│ Australia).jpg                        │              │         │
╘═══════════════════════════════════════╧══════════════╧═════════╛

Frequency

-f daily
-f weekly (spaced evenly throughout the week if less than 7 tabs in the category)
-f monthly (spaced evenly throughout the month if less than 30 tabs in the category)
-f quarterly (spaced evenly throughout 3 months if less than 90 tabs in the category)
-f yearly (spaced evenly throughout the year if less than 365 tabs in the category)

ie. if 14 tabs, two URLs are opened per day of the week

You can pipe stuff

lowcharts

$ wt-dev -p f -col time_created | lowcharts timehist -w 80
Matches: 445183.
Each ∎ represents a count of 1896
[2022-04-13 03:16:05] [151689] ∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
[2022-04-19 07:59:37] [ 16093] ∎∎∎∎∎∎∎∎
[2022-04-25 12:43:09] [ 12019] ∎∎∎∎∎∎
[2022-05-01 17:26:41] [ 48817] ∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
[2022-05-07 22:10:14] [ 36259] ∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
[2022-05-14 02:53:46] [  3942] ∎∎
[2022-05-20 07:37:18] [  2371] ∎
[2022-05-26 12:20:50] [   517]
[2022-06-01 17:04:23] [  4845] ∎∎
[2022-06-07 21:47:55] [  2340] ∎
[2022-06-14 02:31:27] [   563]
[2022-06-20 07:14:59] [ 13836] ∎∎∎∎∎∎∎
[2022-06-26 11:58:32] [  1905] ∎
[2022-07-02 16:42:04] [  1269]
[2022-07-08 21:25:36] [  3062] ∎
[2022-07-15 02:09:08] [  9192] ∎∎∎∎
[2022-07-21 06:52:41] [ 11955] ∎∎∎∎∎∎
[2022-07-27 11:36:13] [ 50938] ∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
[2022-08-02 16:19:45] [ 70973] ∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
[2022-08-08 21:03:17] [  2598] ∎

TODOs (PRs welcome)

• tube: why nan instead of null ?
• all: automatic drop low value indexes? (reduce db size)
• all: Documentation in wiki
• all: more test coverage
• all: investigate fts using sqlite-utils
• all: follow yt-dlp print arg syntax
• all: follow fd-find size arg syntax
• tube: Download subtitle to embed in db tags for search
• fs: split_by_silence without modifying files