Skip to main content
This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

Zc buildout recipe that provides a backup script

Project Description

Code repository: https://ingeniweb.svn.sourceforge.net/svnroot/ingeniweb/iw.recipe.backup/

Change history

trunk (2009-11-25)

  • xxx [Ingeniweb]

0.2.1 (2008-11-24)

  • added ‘base-path’ option [fdupre] if set, use it instead buildout-dir, really usefull if datas are stored in another place in the system (NAS, NFS mount…).

0.2.0 (2008-05-08)

  • added ‘archive-before-backup’ option. [tdesvenain] True by default. If set to 0, buildout is not backuped before a restore
  • added ‘include-folders’ option, that allows to limit backup on some directories [tdesvenain]
  • create backups dir if not exists [tdesvenain]
  • add format option instead of archive-root-name [gawel]
  • fixing test so it passes on all machines [tarek]

0.1.2 (2008-04-11)

  • Added the fs-location option, that allows doing a safe backup of a running Data.fs. [tarek]

0.1.1 (2008-03-21)

  • Added the exclude-folders option
  • Now asks a question before restoring. It is more a conveniency since everything is backed up in any case. [tarek]

0.1.0 (2008-03-12)

  • Initial implementation. [tarek]
  • Created recipe with ZopeSkel [Ingeniweb].

Detailed Documentation

Supported options

The recipe supports the following options:

backup-script-name
Name of the backup script. Default: backup
restore-script-name
Name of the restore script. Default: restore
format
Formated name of the archive. Default to %(name)s-%(year)s-%(month)s-%(day)s-%(hour)s-%(minute)s where name is the section name
exclude-folders
Names of folder to avoid backing up. Relative to buildout root.
include-folders
If set, back up those folders only. Relative to buildout root.
fs-location
If given, indicates the path of the file system storage. The recipe will carefully copy it when doing a backup, by using a transaction-level read of the file. This means that you can launch a backup without stopping the Zope server.
target-folder
Folder where the archives are stored. Mandatory
log-file
File where all calls are recorded. Mandatory
archive-before-restore
If not set to 0, do a complete backup before a restore
prompt-before-restore
If not set to 0, prompt before a restore
base-path
If set, backup this path instead base on instance path

Example usage

We’ll start by creating a buildout that uses the recipe:

>>> import os
>>> root =  os.path.split(sample_buildout)[0]
>>> if root == '':
...     root = '.'

Let’s copy a real Data.fs in our buildout:

>>> import shutil
>>> data_fs = os.path.join(test_dir, 'Data.fs')
>>> shutil.copyfile(data_fs, join(sample_buildout, 'Data.fs'))

>>> write('buildout.cfg',
... """
... [buildout]
... parts = backup
... index = http://pypi.python.org/simple
...
... [backup]
... recipe = iw.recipe.backup
...
... format = %(format)s
... target-folder = %(root)s
... log-file = %(root)s/backup.log
... fs-location = ${buildout:directory}/Data.fs
... """ % {'root': root, 'format':'%(name)s-%(year)s-%(month)s-%(day)s'})

Let’s run the buildout:

>>> null = system(buildout)

Let’s see what we got in the backup script:

>>> print open(join(sample_buildout, 'bin', 'backup')).read()
#!...
<BLANKLINE>
import sys
sys.path[0:0] = [
  ...
  ]
<BLANKLINE>
import iw.recipe.backup.archive
<BLANKLINE>
if __name__ == '__main__':
    iw.recipe.backup.archive.archive_buildout(('/sample-buildout', '%(name)s-%(year)s-%(month)s-%(day)s', '..._TEST_', '/backup.log', [], [], '/sample-buildout/Data.fs', {'name': 'backup'}))
<BLANKLINE>

Ok, let’s call it to backup the current buildout:

>>> print system(join(sample_buildout, 'bin', 'backup'))
Starting the backup...
Archived in /backup-XXXX-XX-XX.tar.gz
...
<BLANKLINE>

We should have a log file generated as well:

>>> print open(join(root, 'backup.log')).read()
20... Starting the backup...
20... INFO Archived in /backup-XXXX-XX-XX.tar.gz

We also have a restore feature:

>>> print system(join(sample_buildout, 'bin', 'restore'))
Usage: ...restore archive_name
<BLANKLINE>

Let’s set the user input:

>>> from iw.recipe.backup.testing import set_input
>>> set_input('Y')

Oh right, the restore script takes the name of the archive:

>>> import glob
>>> arc = glob.glob('%s/*.tar.gz' % root)[0]
>>> print system(join(sample_buildout, 'bin', 'restore %s' % arc))
Are you sure you want to restore ? Every data will be lost ! (y/N)  Y
Archiving current folder before restoring
Starting the backup...
Archived in /before-restore-XXXX-XX-XX-XX-XX.tar.gz
Starting the restore...
Archive restored in /sample-buildout
...
<BLANKLINE>

And a restore always makes an archive on the current folder before it is applied, to make sure nothing is never lost.

There’s also something quite important: make sure the archive and log files are not located in the buildout !:

>>> write('buildout.cfg',
... """
... [buildout]
... parts = backup
... index = http://pypi.python.org/simple
...
... [backup]
... recipe = iw.recipe.backup
...
... target-folder = %(root)s
... log-file = %(root)s/backup.log
... """ % {'root': sample_buildout})

>>> print system(buildout)
Uninstalling backup.
Installing backup.
While:
  Installing backup.
<BLANKLINE>
An internal error occured due to a bug in either zc.buildout or in a
recipe being used:
...
ValueError: Cannot backup within the buildout ! Check your values
<BLANKLINE>

A bit of cleaning:

>>> import glob
>>> arc = glob.glob('%s/*.tar.gz' % root)
>>> for f in arc:
...     os.remove(f)

We can also exclude some folders from being archived:

>>> os.mkdir(join(sample_buildout, 'not'))
>>> open(join(sample_buildout, 'not', 'f'), 'w').write('me file')

>>> os.mkdir(join(sample_buildout, 'neh'))

>>> write('buildout.cfg',
... """
... [buildout]
... parts = backup
... index = http://pypi.python.org/simple
...
... [backup]
... recipe = iw.recipe.backup
...
... target-folder = %(root)s
... log-file = %(root)s/backup.log
... fs-location = ${buildout:directory}/Data.fs
... exclude-folders =
...     %(sample_buildout)s/not
...     %(sample_buildout)s/neh
... """ % {'root': root, 'sample_buildout': sample_buildout})

Running the buildout again:

>>> print system(buildout+' -D')
Installing backup.
Generated script '...backup'.
Generated script '...restore'.
<BLANKLINE>

Let’s backup:

>>> print system(join(sample_buildout, 'bin', 'backup'))
Starting the backup...
Archived in /backup-XXXX-XX-XX-XX-XX.tar.gz
...
<BLANKLINE>

Let’s remove the folder and the Data.fs:

>>> import shutil
>>> shutil.rmtree(join(sample_buildout, 'not'))
>>> os.rmdir(join(sample_buildout, 'neh'))
>>> os.remove(join(sample_buildout, 'Data.fs'))

Let’s restore:

>>> arc = glob.glob('%s/*.tar.gz' % root)[0]
>>> print system(join(sample_buildout, 'bin', 'restore %s' % arc))
Are you sure you want to restore ? Every data will be lost ! (y/N)  Y
...
<BLANKLINE>

And make sure the not folder is not back !

>>> os.path.exists(join(sample_buildout, 'not'))
False
>>> os.path.exists(join(sample_buildout, 'neh'))
False

And the Data.fs is back, so we’re OK:

>>> os.path.exists(join(sample_buildout, 'Data.fs'))
True

We can also restrict archive on some folders

>>> os.mkdir(join(sample_buildout, 'not'))
>>> open(join(sample_buildout, 'not', 'f'), 'w').write('me file')

>>> os.mkdir(join(sample_buildout, 'neh'))
>>> open(join(sample_buildout, 'neh', 'f'), 'w').write('me file')

>>> os.mkdir(join(sample_buildout, 'neh/not'))
>>> open(join(sample_buildout, 'neh/not', 'f'), 'w').write('me file')

>>> write('buildout.cfg',
... """
... [buildout]
... parts = backup
... index = http://pypi.python.org/simple
...
... [backup]
... recipe = iw.recipe.backup
...
... target-folder = %(root)s
... log-file = %(root)s/backup.log
... fs-location = ${buildout:directory}/Data.fs
... exclude-folders =
...     %(sample_buildout)s/not
...     %(sample_buildout)s/neh/not
... include-folders =
...     %(sample_buildout)s/neh
... prompt-before-restore = 0
... archive-before-restore = 0
... """ % {'root': root, 'sample_buildout': sample_buildout})

Running the buildout again:

>>> print system(buildout+' -D')
Uninstalling backup.
Installing backup.
Generated script '...backup'.
Generated script '...restore'.
<BLANKLINE>

Let’s see what we got in the backup script:

>>> print open(join(sample_buildout, 'bin', 'backup')).read()
#!...
<BLANKLINE>
import sys
sys.path[0:0] = [
  ...
  ]
<BLANKLINE>
import iw.recipe.backup.archive
<BLANKLINE>
if __name__ == '__main__':
    iw.recipe.backup.archive.archive_buildout(('/sample-buildout', '...', '..._TEST_', '/backup.log', ['/sample-buildout/not', '/sample-buildout/neh/not'], ['/sample-buildout/neh'], '/sample-buildout/Data.fs', {'name': 'backup'}))
<BLANKLINE>

Let’s backup:

>>> print system(join(sample_buildout, 'bin', 'backup'))
Starting the backup...
Archived in /backup-XXXX-XX-XX-XX-XX.tar.gz
<BLANKLINE>

Let’s remove the folder and the Data.fs:

>>> import shutil
>>> shutil.rmtree(join(sample_buildout, 'not'))
>>> shutil.rmtree(join(sample_buildout, 'neh'))
>>> os.remove(join(sample_buildout, 'Data.fs'))

Let’s restore:

>>> arc = glob.glob('%s/*.tar.gz' % root)[0]
>>> rest = os.popen(join(sample_buildout, 'bin', 'restore %s' % arc))
And make sure the not folder and ‘neh/not’ are not back and neh is back !
>>> os.path.exists(join(sample_buildout, 'not'))
False
>>> os.path.exists(join(sample_buildout, 'neh/not'))
False
>>> os.path.exists(join(sample_buildout, 'neh'))
True

Contributors

Ingeniweb, Author

Release History

Release History

This version
History Node

0.2.1

History Node

0.2.0

History Node

0.1.2

History Node

0.1.1

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
iw.recipe.backup-0.2.1-py2.4.egg (22.6 kB) Copy SHA256 Checksum SHA256 2.4 Egg Nov 25, 2009
iw.recipe.backup-0.2.1.tar.gz (12.9 kB) Copy SHA256 Checksum SHA256 Source Nov 25, 2009

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting