borg-queen 0.1.0b1

Wrapper for Borg Backup.

Queen

A wrapper around Borg backup.

Like other borg wrappers, queen defines a site config file format that specifies what contents should go in a borg archive. It also defines a standard location in /etc/ where it will look for these site config files automatically. Unlike other wrappers, it can run some predefined actions (called plugins), that generate files to be included in the backup.

Currently, there are three plugins:

• 'postgres' that creates binary dumps using pg_dump
• 'mysql' that creates sql dumps using mysqldump
• 'docker_volume' that creates tarballs out of docker volumes

More plugins may be defined in the future.

Installation

Install using pip.

You must also have borg installed and available on the PATH.

If you use the postgres plugin, you also need pg_dump installed.

If you use the mysql plugin, you also need mysqldump installed.

If you use the docker_volume plugin, you also need docker installed.

Usage

usage: queen [-h] [-s SITE_CONFIG]

Take backups using Borg

optional arguments:
  -h, --help            show this help message and exit
  -s SITE_CONFIG, --site-config SITE_CONFIG
                        Backup using the supplied config file instead of those
                        in /etc/queen/sites.d/

Unless supplied with -s, queen will process all site configs in /etc/queen/sites.d/.

Configuration

Queen reads its configuration from /etc/queen/queen.toml

repo_prefix = 'some@repo:prefix'
sentry_dsn = 'some sentry dsn'

[logging]
directory = '/some/directory/'

[borg_env]
BORG_RELOCATED_REPO_ACCESS_IS_OK = 'yes'

[[plugins.postgres]]
host = 'localhost'

repo_prefix, sentry_dsn and logging.directory are required.

The repo location will be a subdirectory inside repo_prefix named after your site config's project_name. You can pass a remote repository URL to create your repo in a remote host, such as user@host:/path. The path specified in repo_prefix must already exist.

The borg_env table may contain environment variables. These will be used for the call to borg create.

The plugins table may contain a table of tables. Keys under plugins must correspond to plugin names. Key inside each plugin will be merged with each site's plugin configs, with the site's values taking precedence.

Site config example

Site configs should be in toml and placed in /etc/queen/sites.d/

project_name = 'myproject'
passphrase = 'some secret'
paths = [
	'/srv/python/myproject',
	'/etc/myproject'
]

[[plugins.postgres]]
database = 'myproject'

[[plugins.mysql]]
database = 'myproject'
user = 'dbuser'
password = 'dbpassword'

Some notes:

• project_name and passphrase are required
• paths should be absolute
• If any of the paths does not exist, queen will emit a WARNING and create a borg archive with the rest of the configuration (existing paths and plugins)

Plugin configuration

Plugins may be configured both globally and per site. When both configurations are present, they are merged with the site's configuration taking precedence.

For example, if you want to use a specific username & password for backing up postgres databases, you may specify them in the global configuration's plugins.postgres table. You can then specify the rest of the configuration (eg. database) in each site without repeating the username & password. If a single still site needs to override the username & password, you can specify it on just that site and they will override the ones in the global configuration.

Postgres

plugins.postgres should be a list of tables, each one describing a database. Each table may contain the following keys:

• database
• user
• password
• host
• port
• pg_dump

Only database is required, all other keys are optional.

user, password, host and port will be used to connect to postgres.

pg_dump must be a list of strings and will be used as the binary to use for taking backups. It is useful for specifying wrapper scripts for pg_dump, for example:

• ['sudo', '-iu', 'postgres', 'pg_dump'] for changing the unix user that will invoke pg_dump.
• ['docker', 'exec', 'container_name', 'pg_dump'] to take a backup from a running postgres container.
• Your own custom shell scripts, wrapping pg_dump

It must be compatible with the actual pg_dump binary and it defaults to ['pg_dump'].

For backwards compatibility, plugins.postgres may be a list of strings, which are interpreted as database names.

Sample config

Full config:

# ...
[[plugins.postgres]]
database = 'database_1'
user = 'user'
password = 'password'
host = 'host'
port = 1234
pg_dump = ['sudo', '-iu', 'postgres', 'pg_dump']

[[plugins.postgres]]
database = 'database_2'
user = 'user'
password = 'password'
host = 'host'
port = 1234

Legacy config:

[plugins]
postgres = [
	'database_1',
	'database_2'
]

MySQL

plugins.mysql should be a list of tables, each one describing a database. Each table may contain the following keys:

• database
• user
• password
• host
• port

Only database is required, all other keys are optional.

Sample config

[[plugins.mysql]]
database = 'myproject'
host = '127.0.0.1'
port = 1234
user = 'dbuser'
password = 'dbpassword'

docker volume

plugins.docker_volume should be a list of tables, each one describing a docker volume. Each table may contain the following keys:

• name: the name of the docker volume. Required.
• image: the image that will be used for making the tarball. Defaults to alpine

Sample config

[[plugins.docker_volume]]
name = 'my_volume'

[[plugins.docker_volume]]
name = 'my_other_volume'
image = 'debian:bullseye-slim'

Repo/Archive Format

Queen is going to use your project as the repo name and create it if it does not exist. Each Archive will be named as the ISO-8601 timestamp of the time the backup was taken in UTC (unlike borg's {utcnow} it includes a timezone).

Inside each archive, your paths are included as is. Plugins place their content inside _queen/{plugin_name}.

Using the config above as an example your archive would contain:

srv/
	python/
		myproject/
			...
etc/
	myproject/
		...
_queen/
	postgres/
		myproject.pgdump
	mysql/
		myproject.dump.sql