Skip to main content
Donate to the Python Software Foundation or Purchase a PyCharm License to Benefit the PSF! Donate Now

DigitalOcean snapshot automation service

Project description

goutte

DigitalOcean doesn't propose any way of automating snapshots. There are some SaaS that can take care of it but paying to execute some API requests seemed a bit off.

That's why we developed a simple script which can run with cron jobs or in CI services like Travis for free. We use it daily to manage our backups.

It includes:

  • Snapshoting droplets
  • Snapshoting volumes
  • Retention policy
  • Pruning snapshots

Requirements

  • Python ^3.6
  • A DigitalOcean account

Installation

Install it directly from pip:

pip3 install --user goutte

Configuration file

Goutte takes its configuration from a pretty straightforward toml file. We provided and example in goutte.example.toml.

retention = 10     # Number of backups to keep per droplet/volume

[droplets]
names = [          # Array of droplets you want to snapshot
  'server01',
  'server02',
  'server03',
]

[volumes]
names = [          # Array of volumes you want to snapshot
  'db01',
  'redis01',
  'redis02',
]

Usage

Goutte takes two arguments which can also be set via environment variables:

# Help Description Environment variable
1 CONFIG Path to the toml configuration file GOUTTE_CONFIG
2 DO_TOKEN Your DigitalOcean API token GOUTTE_DO_TOKEN
Usage: goutte [OPTIONS] CONFIG DO_TOKEN

  DigitalOcean snapshots automation.

Options:
  --only [snapshot|prune]  Only snapshot or only prune
  --debug                  Enable debug logging
  --version                Show the version and exit.
  --help                   Show this message and exit.

Running "snapshot only" for a configuration file containing one droplet and one volume:

$ goutte goutte.toml $do_token --only snapshot
13:32:48 - INFO - Starting goutte v1.0.1
13:32:52 - INFO - sgp1-website-01 - Snapshot (goutte-sgp1-website-01-20181220-56bde)
13:32:59 - INFO - sgp1-mariadb-01 - Snapshot (goutte-sgp1-mariadb-01-20181220-3673d)

Run with Docker

We have a Docker image ready for you to use on Docker Hub. It will read by default the configuration under /goutte/goutte.toml

docker run \
  -e GOUTTE_DO_TOKEN=${do_token} \
  -v $(pwd)/goutte.toml:/goutte/goutte.toml \
  tomochain:goutte

Automating with Travis

You can easily automate it via cron job but the easiest way would be by leveraging free CI tools like Travis.

  1. You can create a repo which contains your goutte.toml configuration and the following travis file .travis.yml :
language: python
python: 3.6

install:
  - pip install goutte

script:
  - goutte goutte.toml # Don't forget to set GOUTTE_DO_TOKEN in Travis config
  1. Enable the repo in Travis and then go to the configuration
  2. Add the environment variable GOUTTE_DO_TOKEN with the value of your DigitalOcean API key
  3. Enable daily cron job
  4. You're good to go, goutte will run everyday and take care of the snapshots.

Note: You can have different retentions for different volumes by having multiple configurations.

# ...
script:
  - goutte 10days.toml
  - goutte 1day.toml

You can see how we set it up for ourself here.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
goutte-1.0.1-py3-none-any.whl (48.7 kB) Copy SHA256 hash SHA256 Wheel py3
goutte-1.0.1.tar.gz (17.2 kB) Copy SHA256 hash SHA256 Source None

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page