An exporter written in python to export all documents from a bookstack instance in different formats
Project description
bookstack-file-exporter
Background
Features are actively being developed. See Future Items
section for more details. Open an issue for a feature request.
This tool provides a way to export Bookstack pages in a folder-tree layout locally with an option to push to remote object storage locations. See Backup Behavior
section for more details on how pages are organized.
This small project was mainly created to run as a cron job in k8s but works anywhere. This tool allows me to export my docs in markdown, or other formats like pdf. I use Bookstack's markdown editor as default instead of WYSIWYG editor and this makes my notes portable anywhere even if offline.
The main use case is to backup all docs in a folder-tree format to cover the scenarios:
- Offline copy wanted.
- Back up at a file level as an accessory or alternative to disk and volume backups.
- Share docs with another person to keep locally.
- Migrate to Markdown documenting for simplicity.
- Provide an easy way to do automated file backups locally, in docker, or kubernetes.
Supported backup targets are:
- local
- minio
- s3 (Not Yet Implemented)
Supported backup formats are shown here and below:
- html
- markdown
- plaintext
Backups are exported in .tgz
format and generated based off timestamp. Export names will be in the format: %Y-%m-%d_%H-%M-%S
(Year-Month-Day_Hour-Minute-Second). Files are first pulled locally to create the tarball and then can be sent to object storage if needed. Example file name: bookstack_export_2023-09-22_07-19-54.tgz
.
The exporter can also do housekeeping duties and keep a configured number of archives and delete older ones. See keep_last
property in the Configuration
section. Object storage provider configurations include their own keep_last
property for flexibility.
Using This Application
Ensure a valid configuration is provided when running this application. See Configuration
section for more details.
Run via Pip
python -m pip install bookstack-file-exporter
# if you already have python bin directory in your path
bookstack-file-exporter -c <path_to_config_file>
# using pip
python -m bookstack_file_exporter -c <path_to_config_file>
Command line options:
option | required | description |
---|---|---|
-c , --config-file |
True | Relative or Absolute path to a valid configuration file. This configuration file is checked against a schema for validation. |
-v , --log-level |
False, default: info | Provide a valid log level: info, debug, warning, error. |
Note: This application is tested and developed on Python 3.11.X
. It will probably work for >= 3.8
but is recommended to install (or set up a venv) a 3.11.X
version.
Run Via Docker
Example:
docker run \
--user ${USER_ID}:${USER_GID} \
-v $(pwd)/config.yml:/export/config/config.yml:ro \
-v $(pwd)/bkps:/export/dump \
homeylab/bookstack-file-exporter:latest
Minimal example with object storage upload:
docker run \
-v $(pwd)/config.yml:/export/config/config.yml:ro \
homeylab/bookstack-file-exporter:latest
Tokens and other options can be specified, example:
# '-e' flag for env vars
# --user flag to override the uid/gid for created files
docker run \
-e LOG_LEVEL='debug' \
-e BOOKSTACK_TOKEN_ID='xyz' \
-e BOOKSTACK_TOKEN_SECRET='xyz' \
--user 1000:1000 \
-v $(pwd)/config.yml:/export/config/config.yml:ro \
-v $(pwd)/bkps:/export/dump \
homeylab/bookstack-file-exporter:latest
Bind Mounts:
purpose | static docker path | description | example |
---|---|---|---|
config |
/export/config/config.yml |
A valid configuration file | -v /local/yourpath/config.yml:/export/config/config.yml:ro |
dump |
/export/dump |
Directory to place exports. This is optional when using remote storage option(s). Omit if you don't need a local copy. | -v /local/yourpath/bkps:/export/dump |
Authentication
Note visibility of pages is based on user, so use a user that has access to pages you want to back up.
Ref: https://demo.bookstackapp.com/api/docs#authentication
Provide a tokenId and a tokenSecret as environment variables or directly in the configuration file.
BOOKSTACK_TOKEN_ID
BOOKSTACK_TOKEN_SECRET
Env variables for credentials will take precedence over configuration file options if both are set.
For object storage authentication, find the relevant sections further down in their respective sections.
Configuration
See below for an example and explanation. Optionally, look at examples/
folder of the github repo for more examples.
For object storage configuration, find more information in their respective sections
Schema and values are checked so ensure proper settings are provided. As mentioned, credentials can be specified as environment variables instead if preferred.
# if http/https not specified, defaults to https
# if you put http here, it will try verify=false, to not check certs
host: "https://bookstack.yourdomain.com"
# You could optionally set the bookstack token_id and token_secret here instead of env
# If env variable is also supplied, env variable will take precedence
credentials:
token_id: ""
token_secret: ""
# additional headers to add, examples below
additional_headers:
test: "test"
test2: "test2"
User-Agent: "test-agent"
# supported formats from bookstack below
# valid formats: markdown, html, pdf, plaintext
# you can specify one or as many as you'd like
formats:
- markdown
- html
- pdf
- plaintext
# optional minio configuration
# If not required, you should omit/comment out the section
# You can specify env vars instead for access and secret key
# See Minio Backups section of this doc for more info on required fields
minio_config:
host: "minio.yourdomain.com"
access_key: ""
secret_key: ""
region: "us-east-1"
bucket: "mybucket"
path: "bookstack/file_backups"
keep_last: 5
# output directory for the exported archive
# relative or full path
# User who runs the command should have access to write and create sub folders in this directory
# optional, if not provided, will use current run directory by default
output_path: "bkps/"
# optional export of metadata about the page in a json file
# this metadata contains general information about the page
# like: last update, owner, revision count, etc.
# omit this or set to false if not needed
export_meta: true
# optional if specified exporter can delete older archives
# valid values are:
# set to -1 if you want to delete all archives after each run
# - this is useful if you only want to upload to object storage
# set to 1+ if you want to retain a certain number of archives
# set to 0 or comment out section if you want no action done
keep_last: 5
Backup Behavior
We will use slug names (from Bookstack API) by default, as such certain characters like !
, /
will be ignored and spaces replaced.
All sub directories will be created as required during the export process.
Shelves --> Books --> Chapters --> Pages
## Example
kafka
---> controller
---> settings
---> logs (chapter)
---> retention.md
---> compression.pdf
---> something.html
---> other.txt
---> optional
---> main
---> deploy
---> broker
---> settings
---> deploy
---> schema-registry
---> protobuf
---> settings
Books without a shelf will be put in a shelve folder named unassigned
.
Empty/New Pages will be ignored since they have not been modified yet from creation and are empty but also do not have a valid slug. Example:
{
...
"name": "New Page",
"slug": "",
...
}
You may notice some directories (books) and/or files (pages) in the archive have a random string at the end, example - nKA
: user-and-group-management-nKA
. This is expected and is because there were resources with the same name created in another shelve and bookstack adds a string at the end to ensure uniqueness.
Minio Backups
Optionally, look at examples/
folder of the github repo for more examples.
minio_config:
# a host/ip + port combination is also allowed
# example: "minio.yourdomain.com:8443"
host: "minio.yourdomain.com"
# this is required since minio api appears to require it
# set to the region your bucket resides in
# if unsure, try "us-east-1" first
region: "us-east-1"
# bucket to upload to
bucket "mybucket"
# access key for the minio instance
# optionally set as env variable instead
access_key: ""
# secret key for the minio instance
# optionally set as env variable instead
secret_key: ""
# the path of the backup
# optional, will use root bucket path if not set
# in example below, the exported archive will appear in: `<bucket_name>:/bookstack/backups/bookstack-<timestamp>.tgz`
path: "bookstack/file_backups"
# optional if specified exporter can delete older archives
# valid values are:
# set to 1+ if you want to retain a certain number of archives
# set to 0 or comment out section if you want no action done
keep_last: 5
As mentioned you can optionally set access and secret key as env variables. If both are specified, env variable will take precedence.
MINIO_ACCESS_KEY
MINIO_SECRET_KEY
Future Items
- Be able to pull media/photos locally and place in their respective page folders for a more complete file level backup.
- Include the exporter in a maintained helm chart as an optional deployment. The helm chart is here.
- Export S3 or more options.
- Filter shelves and books by name - for more targeted backups. Example: you only want to share a book about one topic with an external friend/user.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for bookstack-file-exporter-0.0.2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | e4f83aae14738b76b326df669304640a5c2c7b1a691d596e566f75cdedede67f |
|
MD5 | 2deadba251257c5ace233e261752ae9e |
|
BLAKE2b-256 | c10ffb5f96b267584f40309b26b3d24f0e33bcd6e2aaac6848afa46a7679b865 |
Hashes for bookstack_file_exporter-0.0.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cef2a5e4d6b6961e02fc5b674ac54d3647315c0c11c065a01c4bda7f9b8e66dd |
|
MD5 | a5daa7ecd880286b5aeff87cf44b058c |
|
BLAKE2b-256 | 4bf4cc077ef0dc41cb9568098f0d8156c56cde6c9cdf5e3d559878a0198d96ed |