szurubooru-toolkit 0.6.0

Python package and script collection to manage szurubooru.

szurubooru-toolkit

Python package and script collection to manage your szurubooru image board.

Requirements

In order to run the included scripts, a Python release >=3.8 and the configuratrion file config.toml is required.

The config.toml file needs to be always present in your current working directory from where you are executing the scripts. You can find a sample config file in the GitHub repository of this package.

Installation

This package is available on PyPI and can be installed with pip: pip install szurubooru-toolkit

Alternatively, you can clone the package from GitHub and set everything up with Poetry. In the root directory of this repository, execute poetry install.

Please note this package requires Tensorflow for Deepbooru tagging and other required packages which do take up quite a lot of disk space (>400MB). A future release of this package will offer an option without the need to install Tensorflow if you don't want to use Deepbooru tagging.

User configuration

Make your changes in the config_sample.toml file provided in the git repo and rename it to config.toml afterwards.

Note that path names have to be specified with forward slashes (/) if you're using Windows.

config.toml reference

Section	Option	Description	Examples/Default
szurubooru	url	The URL of your szurubooru	"https://szuru.example.com"
szurubooru	username	Username which connects to the szuruboori API	"my_szuru_user"
szurubooru	api_token	API token of username. Generate one in szurubooru from Account > Login tokens > Create token	"my_api_token"
szurubooru	public	If your szurubooru is reachable over the internet	false
auto_tagger	saucenao_api_token	In case you want to increase your daily query limit	"my_saucenao_api_token"
auto_tagger	saucenao_enabled	Set this to false and deepbooru_enabled to true if you only want to tag with Deepbooru	true
auto_tagger	deepbooru_enabled	If enabled, tag the post with Deepbooru if no tags with SauceNAO were found	false
auto_tagger	deepbooru_model	Path to the Deepbooru model	"./misc/deepbooru/model-resnet_custom_v3.h5"
auto_tagger	deepbooru_threshold	Define how accurate the matched tag from Deepbooru has to be	"0.7"
auto_tagger	deepbooru_forced	Always tag with SauceNAO and Deepbooru	false
auto_tagger	hide_progress	Set this to true to hide the progress bar	false
auto_tagger	tmp_path	Local path where media files get downloaded temporarily if you szurubooru is not public.	/tmp, C:/Users/Foo/Desktop
danbooru	user	Danbooru user	"None"
danbooru	api_key	Danbooru api key	"None"
gelbooru	user	Gelbooru user	"None"
gelbooru	api_ley	Gelbooru api key	"None"
konachan	user	Konachan user	"None"
konachan	password	Konachan password	"None"
yandere	user	Yandere user	"None"
yandere	password	Yandere password	"None"
pixiv	user	Pixiv user. Currently not being used.	"None"
pixiv	password	Pixiv password. Currently not being used.	"None"
pixiv	token	Pixiv token. Currently not being used.	"None"
upload_media	src_path	Every valid media file under this dir (recursively) will get uploaded	"/local/path/to/upload/dir"
upload_media	hide_progress	Set this to true to hide the progress bar	false
upload_media	cleanup	Set this to true if images in the src_path should be deleted after upload	false
upload_media	tags	These tags will get set for all uploaded posts. Separate them by a comma.	["tagme", "tag1", "tag2", "tagN"]
upload_media	auto_tag	Set this to true if you want your post to be automatically tagged after upload	false
upload_media	max_similarity	Adjust this value to ignore posts if a similar post higher than the threshold has already been uploaded	"0.99"
upload_media	convert_to_jpg	Convert images to JPG to save disk space. This won't overwrite the source files and only affects the uploaded image.	false
upload_media	convert_threshold	Only images above this threshold will be converted to jpg if convert_to_jpg is True.	"3MB|500KB"
import_from_booru	deepbooru_enabled	Apply Deepbooru tagging additionally besides fetched tags from Booru	false
import_from booru	hide_progress	Set this to true to hide the progress bar	false
create_tags	hide_progress	Set this to true to hide the progress bar	false
delete_posts booru	hide_progress	Set this to true to hide the progress bar	false
reset_posts booru	hide_progress	Set this to true to hide the progress bar	false
tag_posts	hide_progress	Set this to true to hide the progress bar	false
logging	log_enabled	If logging to a log file should be enabled	false
logging	log_file	Specify the path of the log file	"C:/Users/Foo/Desktop/szurubooru_toolkit.log"
logging	log_level	Specify the log level. DEBUG logs the most information	"DEBUG"|"INFO"|"WARNING"|"ERROR"|"CRITICAL"
logging	log_colorized	If the log file should be colorized. Requires compatible viewer (e.g. less -r <log_file>).	true

Creating a SauceNAO account and an API key is recommended. Please consider supporting the SauceNAO team as well by upgrading your plan. With a free plan, you can request up to 200 posts in 24h.

For Deepbooru support, download the current release here (v3-20211112-sgd-e28) and extract the contents of the zip file. Specify the path of the folder with the extracted files in deepbooru_model. Please note that you have to set deepbooru_enabled if you want to use it.

Scripts

Following scripts are currently available:

• auto-tagger: Batch tagging of posts with SauceNAO and Deepbooru
• create-tags: Batch creation of tags with their categories
• delete-posts: Batch delete of posts
• import-from-booru: Batch importing of posts with their tags from various Boorus
• reset-posts: Batch resetting of posts (remove tags and sources)
• upload-media: Batch upload of media files from local source folder
• tag-posts: Manual batch tagging

See the descriptions below on how to use them.

If you installed this package with pip, you can generally just call the scripts from your shell (if your $PATH is set correctly).

If you cloned the repo from GitHub, prefix the above scripts with poetry run, e.g. poetry run auto-tagger "date:today". Note that your current working directory has to be the the root of the GitHub project.

auto-tagger

This script accepts a szurubooru query as a user input, fetches all posts returned by it and attempts to tag it using SauceNAO/Deepbooru.

If no matches from SauceNAO were found, the script keeps the previously set tags of the post and additionally appends the tag tagme.

You can set deepbooru_enabled to true in your config.toml file. In that case, the script falls back to tag posts with the supplied Deepbooru model. If you only want to use Deepbooru, set deepbooru_enabled to true and saucenao_enabled to false. If you want to use SauceNAO and Deepbooru, set following options to true: saucenao_enabled, deepbooru_enabled and deepbooru_forced.

Usage

usage: auto-tagger [-h] [--sankaku_url SANKAKU_URL] [--add-tags ADD_TAGS] [--remove-tags REMOVE_TAGS] query

This script will automagically tag your szurubooru posts based on your input query.

positional arguments:
  query                 Specify a single post id to tag or a szuru query. E.g. "date:today tag-count:0"

optional arguments:
  -h, --help            show this help message and exit
  --sankaku_url SANKAKU_URL
                        Fetch tags from specified Sankaku URL instead of searching SauceNAO.
  --add-tags ADD_TAGS   Specify tags, separated by a comma, which will be added to all posts matching your query
  --remove-tags REMOVE_TAGS
                        Specify tags, separated by a comma, which will be removed from all posts matching your query

Examples

• auto-tagger "date:today tag-count:0"
• auto-tagger "date:2021-04-07"
• auto-tagger "tagme"
• auto-tagger "id:100..111"
• auto-tagger "id:100,101"
• auto-tagger --add-tags "foo,bar" --remove-tags "baz" "tagme"

If we want to tag a single post, we can omit the keyword id in our query:

• auto-tagger 100

Alternatively, we can tag a single post and specify --sankaku_url to fetch the tags from the supplied URL (untested with current release):

auto-tagger --sankaku_url https://chan.sankakucomplex.com/post/show/<id> 100

This is especially useful since Sankaku has changed their API and aggregator sites like SauceNAO don't have the latest results there.

import-from-booru

This scripts imports posts and their tags from various Boorus that matched your input query.

In the config.toml file, you can set if the post should be additionally tagged with Deepbooru and if the progress bar should be shown. Since this script is using the upload-media script to upload the post, following settings apply from the upload-media section: convert_to_jpg and convert_threshold.

Usage

usage: import-from-booru [-h] [--limit LIMIT] {danbooru,gelbooru,konachan,yandere,all} query

This script downloads and tags posts from various Boorus based on your input query.

positional arguments:
  {danbooru,gelbooru,konachan,yandere,all}
                        Specify the Booru which you want to query. Use all to query all Boorus.
  query                 The search query for the posts you want to download and tag

optional arguments:
  -h, --help            show this help message and exit
  --limit LIMIT         Limit the search results to be returned (default: 100)

Examples

• import-from-booru danbooru "tag1 tagN"
• import-from-booru yandere "tag1 tag2 -tagN"
• import-from-booru all "tag1 -tagN"

Note that if you specify all to download from all Boorus, you are limited to two tags because free Danbooru accounts are limited to two tags per query. If you have a Gold/Platinum account, set your credentials in config.toml. Note that it's currently untested if the script will work with upgraded accounts.

upload-media

This script searches through your specified upload folder in the config.toml file for any image/video files and uploads them to your szurubooru.

Usage

After editing and renaming the config_sample.toml file to config.toml, we can just execute the script.

If you installed it with pip, execute upload-media. Note that config.toml has to be in your current working directory.

If you cloned the repo from GitHub, execute poetry run upload-media. Note that your current working directory has to be the the root of the GitHub project.

tag-posts

Usage

usage: tag_posts.py [-h] [--add-tags ADD_TAGS] [--remove-tags REMOVE_TAGS] [--mode {append,overwrite}] query

This script will tag your szurubooru posts based on your input arguments and mode.

positional arguments:
  query                 The search query for the posts you want to tag

optional arguments:
  -h, --help            show this help message and exit
  --add-tags ADD_TAGS   Specify tags, separated by a comma, which will be added to all posts matching your query.
  --remove-tags REMOVE_TAGS
                        Specify tags, separated by a comma, which will be removed from all posts matching your query.
  --mode {append,overwrite}
                        Set mode to overwrite to remove already set tags, set append to keep them (default: append).

Examples

• tag-posts --add-tags "foo,bar" "date:2021-04-07"
• tag-posts --add-tags "foo,bar" --mode "overwrite" "foo bar"
• tag-posts --add-tags "foo,bar" --remove-tags "baz" "foo"

reset-posts

Usage

usage: reset_posts.py [-h] [--except-ids EXCEPT_IDS] [--add-tags ADD_TAGS] query

This script will remove tags and sources from your szurubooru posts based on your input search query.

positional arguments:
  query                 The search query for the posts you want to reset.

optional arguments:
  -h, --help            show this help message and exit
  --except-ids EXCEPT_IDS
                        Specify the post ids, separated by a comma, which should not be reset. Example: --except-ids "3,4,5"
  --add-tags ADD_TAGS   Specify tags, separated by a comma, which will be added to all posts matching your query after resetting.

Examples

• reset-posts "foobar"
• reset-posts --add-tags "tagme" "foobar"
• reset-posts --add-tags "tagme,foo" "foobar"
• reset-posts --except-ids "2,4" --add-tags "tagme,foo" "foobar"

delete-posts

Usage

usage: delete_posts.py [-h] [--except-ids EXCEPT_IDS] query

This script will delete your szurubooru posts based on your input search query.

positional arguments:
  query                 The search query for the posts you want to delete.

optional arguments:
  -h, --help            show this help message and exit
  --except-ids EXCEPT_IDS
                        Specify the post ids, separated by a comma, which should not be deleted. Example: --except-ids "3,4,5

Examples

• delete-posts "id:10,11,100,23"
• delete-posts --except-ids "12,23,44" "id:10..50"

create-tags

Usage

usage: create_tags.py [-h] [--tag-file TAG_FILE]

This script will read the tags from specified file and create them in your szurubooru.

optional arguments:
  -h, --help           show this help message and exit
  --tag-file TAG_FILE  Specify the local path to the file containing the tags and categories (default: ./misc/tags/tags.txt)

This script reads per default the file ./misc/tags/tags.txt, parses its contents and creates the tags in your szurubooru. Alternatively, specify --tag-file as an argument with the path to the file containing the tags and categories.

If the tag already exists, it will get skipped.

You can use tools like Grabber to download a tag list from common boorus.

The file has to be in following format:

<tag_a>,<category_number>
<tag_b>,<category_number>
<tag_..n>,<category_number>

Category	Number
default	0
artist	1
series	2
character	3
meta	4

Image credit

GitHub repo icon: Code icons created by Smashicons - Flaticon