Download utility for thetrove.is
Project description
thetrove-downloader
This Python package allows to easily download any resource, be it folder or file, from The Trove.is.
Installation and Requirements
The program requires Python 3.9 or above.
To install the program it is sufficient to use Python pip and get the package thetrove-downloader
. The package will add a new command thetrove-downloader
.
python3 -m pip install thetrove-downloader
Updates should be performed via pip.
python3 -m pip install --upgrade thetrove-downloader
Usage
thetrove-downloader [-h] [-t, --target TARGET] [-j, --json JSON]
[-f, --folder FOLDER] [-o, --output OUTPUT]
[-b, --blacklist BLACKLIST]
[-w, --whitelist WHITELIST] [--no-download]
[--no-preserve-root]
-
-h, --help
show help message -
-t, --target <TARGET>
download target (folder or file)
To download a specific resource, start the program with the--target
command and supply a valid path (e.g. "Books/White Star"). -
-j, --json <JSON>
save/read instructions from a JSON file
The--json
option allows to specify a file to save the target, folder, output, whitelist, and blacklist instructions in JSON format. Launching the program with only the--json
option will load the instructions previously saved in the file and perform all of them in succession. This can be used to keep a local copy of specific resources up to date. -
-f, --folder <FOLDER>
destination folder
The--folder
option allows to specify a folder in which the targeted resource will be downloaded. -
-o, --output <OUTPUT>
output name of download target
The--output
option allows to override the folder/file name of the resource. Note that if the targeted resource is a folder, the option will only apply to the top-level folder and leave all contents therein unaltered. -
-b, --blacklist <BLACKLIST>
regex blacklist for files/folders
The--whitelist
option allows to specify a regex pattern that will be used to skip files and folder that do not contain the supplied pattern. Overwrites blacklist options. -
-w, --whitelist <WHITELIST>
regex whitelist for files/folders, overrides blacklist
The--whitelist
option allows to specify a regex pattern that will be used to skip files and folder that do not contain the supplied pattern. Overwrites blacklist options. -
--no-download
list content without downloading
The--no-download
flag allows to list the content under the supplied target without performing any download. The instructions will still be saved in a JSON file if the--json
options is used, but not the flag. -
--no-preserve-root
allow download of root folders
The--no-preserve-root
flag allows downloading entire root folders such as "/Books/". This flag is not preserved with the--json
option and must be passed manually every time.
NOTE: the program requires to use either --target
, --json
, or both --target
and --json
. Without one of these combinations the program won't have a valid download instruction.
Notes on Crawling
The program performs all GET requests without throttling or delay and with no disallowed paths as per the instructions at thetrove.is/robots.txt.
However, please do take care to not overuse this utility and risk overwhelming their connection. Thank you.
Root folders such as "/Books" are protected and cannot be downloaded without using the --no-preserve-root
flag.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for thetrove-downloader-1.0.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9c2607e1b0b0c188354d42a4f7efce91f50f15cc0da156445c14c04716792991 |
|
MD5 | 3391a2ddb5f9f22666df1f46f4d32777 |
|
BLAKE2b-256 | 4d8be08f6d5803dc8a14f4e4fbc6db8978e91f017b43a3aca7de40ff8b4560e4 |
Hashes for thetrove_downloader-1.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e96b4b59ae0c50dff83f0ada209d1aa580f1bd70e0eaf439c5ebf5ac127f4ebe |
|
MD5 | 99229a532a559f98ac043cc52258ff53 |
|
BLAKE2b-256 | b59da5836d3a254da46bc7e228c6860300281e61384e699ce9bd48992652ee1f |