spare 0.4.0

One S3 backup, encrypted on the fly.

Description

This tool is used at Seantis to create backups stored on various S3 compatible services using a very limited subset of S3 commands.

This is what you need to know:

• Spare should not be used in production yet! Use at your own risk.

• Buckets should be managed by spare exclusively, other files are deleted!

• Each hostname must have a separate bucket for each backed up path.

• Spare stores exactly one copy of your data.

• Files are compressed using LZMA and encrypted on the client using AES-SIV.

• During upload, spare is limited to one core and less than 100MiB of memory.

• Duplicated files are stored only once.

• If you forget your password, you cannot restore your files.

• Spare is meant to be used with configuration management, the cli is minimal.

• Python 3.6.2+ is required.

Usage

To install spare:

pip install spare

To set the connection parameters:

export SPARE_ENDPOINT=host
export SPARE_ACCESS_KEY=access-key
export SPARE_SECRET_KEY=secret-key

To backup a folder:

spare create --password my-password --bucket my-bucket --path /my-path

To restore a backup:

spare restore --password my-password --bucket my-bucket --path /my-path

To verify the backup (downloads everything!):

spare verify --password my-password --bucket my-bucket --path /my-path

Locking

During a spare run, the bucket is locked, so other Spare instances know not to touch it. However, it’s possible that a lock persists when Spare crashes.

If that happens, make sure that the host pushing the backup is truly not running any spare processes, then use the unlock command:

To unlock a backup (if locked)

spare unlock –password my-password –bucket my-bucket

If however, you would like to lock a backup to avoid anyone from changing it, run the following command:

spare lock –password my-password –bucket my-bucket

Run the Tests

Spare uses Minio <https://www.minio.io> to run tests against a real object storage server. As a result it can take a bit for the first test run to complete, as Minio is downloaded and stored in the pytest cache directory:

pip install -e '.[test]'
py.test

Build Status

License

spare is released under the MIT license

Changelog

0.4.0 (2019-07-29)

• Improves handling of SIGINT/SIGTERM.

  It should now be safe to kill Spare during backup runs. Spare will still block until the current object is properly uploaded, but it won’t try to finish the snapshot.

  The uploaded files might of course be discarded, but the bucket should be unlocked and the uploaded files should either be fully uploaded, or not at all. [href]

0.3.0 (2019-05-28)

• Adds a lock/unlock commands. [href]

0.2.6 (2019-05-09)

• Fixes tests failing on Travis, again. [href]

0.2.5 (2019-05-09)

• Fixes tests failing on Travis. [href]

0.2.4 (2019-05-09)

• Stops showing errors when files go missing during operation. [href]

0.2.3 (2018-06-20)

• Fixes large snapshots getting pruned after creation. [href]

0.2.2 (2018-05-25)

• Fixes tests. [href]

0.2.1 (2018-05-25)

• Delay sigterm handling during backup and restore operations. [href]

• Changes ‘–force’ into a flag. [href]

0.2.0 (2018-05-11)

• Adds the inode to the snapshot identity to ensure that a newly setup host doesn’t overwrite existing backups. [href]

0.1.3 (2018-05-10)

• Adds the ability to exclude files from the backup. [href]

0.1.2 (2018-05-10)

• Increases the download speed during restore operations. [href]

• Shows a warning when a user/group could not be found during restore. [href]

• Lowers the timeouts and retry rates. [href]

0.1.1 (2018-05-04)

• Lowers the number of requests needed to upload data. [href]

0.1.0 (2018-04-26)

• Initial Release. [href]