Skip to main content

ZFS synchronization and snapshotting tools

Project description

# ZFS tools

This repo has diverged from the upstream [Rudd-O/zfs-tools](http://github.com/Rudd-O/zfs-tools), as described [here](https://github.com/Rudd-O/zfs-tools/issues/25). It is now being maintained directly here by the original author of the `zbackup` utility, and has its own release numbering.

Differences from upstream include (but may not be limited to):

  • parallel replication, for increased performance of `zreplicate`

  • locking of filesystems during replication, to avoid problems arising from very long replications

  • resumable replications using the facility introduced in zfs 0.7.0

This version of the zfs-tools suite is used in production on the author’s system comprising 7 fileservers, 3400 filesystems, and 550TB of data.

The ZFS backup tools will help you graft an entire ZFS pool as a filesystem into a backup machine, without having to screw around snapshot names or complicated shell commands or crontabs.

The utilities let you do this:

  1. zfs-shell: a shell that allows remote ZFS administration and nothing more

  1. zsnap: a command that snapshots a dataset or pool, then deletes old snapshots

  2. zreplicate a command that replicates an entire dataset tree using ZFS replication streams. Best used in combination with zsnap as in:

    • zsnap on the local machine

    • zreplicate from the local machine to the destination machine

    Obsolete snapshots deleted by zsnap will be automatically purged on the destination machine by zreplicate, as a side effect of using replication streams. To inhibit this, use the –no-replication-stream option.

    Run zreplicate –help for a compendium of options you may use.

  3. zbackup: a command to snapshot and replicate filesystems according to their user properties. This uses zsnap and zreplicate to do the work, which is all driven by properties. For details, see this [further description of zbackup](doc/README-zbackup.md).

  4. zflock: a command to lock a filesystem against replication by zbackup. For details, see this [further description of zbackup](doc/README-zbackup.md).

## Setting up

Setup is rather complicated. It assumes that you already have ZFS running and vaults on both the machine you’re going to back up and the machine that will be receiving the backup.

### On the machine to back up

  • Install the zfs-shell command cp zfs-shell /usr/local/sbin chmod 755 /usr/local/sbin/zfs-shell chown root.root /usr/local/sbin/zfs-shell

  • Create a user with a home directory and shell zfs-shell useradd -rUm -b /var/lib -s /usr/local/sbin/zfs-shell zfs

  • Let sudo know that the new user can run the zfs command zfs ALL = NOPASSWD: /usr/local/sbin/zfs (ensure you remove the requiretty default on /etc/sudoers) (check sudoers.zfs-tools in contrib/ for an example)

  • Set up a cron job to run zsnap as frequently as you want to, snapshotting the dataset you intend to replicate.

### On the backup machine

  • Set up public key authentication for SSH so the backup machine may log as the user zfs (as laid out above) in the machine to be backed up.

  • Create a dataset to receive the backup stream.

  • Set up a cron job to fetch the dataset snapshotted by zsnap from the remote machine into the newly created dataset. You will use zreplicate for that (see below for examples).

  • After the first replication, you may want to set the mountpoint attributes on the received datasets so they do not automount on the backup machine.

### Test

If all went well, you should be able to do this without issue:

(on the machine to back up)

[root@peter] zsnap senderpool

(on the machine to receive)

[root@paul] zfs create receiverpool/senderpool # <— run this ONLY ONCE zreplicate -o zfs@paul:senderpool receiverpool/senderpool # this should send the entire senderpool with all snapshots # over from peter to paul, placing it in receiverpool/senderpool

(on the machine to back up)

[root@peter] zsnap senderpool

(on the machine to receive)

[root@paul] zreplicate -o zfs@paul:senderpool receiverpool/senderpool # this should send an incremental stream of senderpool # into receiverpool/senderpool

And that’s it, really.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

zfs-tools-zbackup-0.5.1.tar.gz (38.3 kB view details)

Uploaded Source

Built Distribution

zfs_tools_zbackup-0.5.1-py2-none-any.whl (33.5 kB view details)

Uploaded Python 2

File details

Details for the file zfs-tools-zbackup-0.5.1.tar.gz.

File metadata

  • Download URL: zfs-tools-zbackup-0.5.1.tar.gz
  • Upload date:
  • Size: 38.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.12.1 pkginfo/1.4.2 requests/2.21.0 setuptools/40.6.3 requests-toolbelt/0.8.0 tqdm/4.28.1 CPython/2.7.5

File hashes

Hashes for zfs-tools-zbackup-0.5.1.tar.gz
Algorithm Hash digest
SHA256 a54f0f9342b8f6ec1a0acf4c0276cd0dec0c43265fbaeba58490ff976b6833ad
MD5 ba56b7dfb346d22732c75112bf4f141a
BLAKE2b-256 0daf7d3361f955f35ad0c8823a43843795e247ea23addb030447907fdf4ae6cb

See more details on using hashes here.

File details

Details for the file zfs_tools_zbackup-0.5.1-py2-none-any.whl.

File metadata

  • Download URL: zfs_tools_zbackup-0.5.1-py2-none-any.whl
  • Upload date:
  • Size: 33.5 kB
  • Tags: Python 2
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.12.1 pkginfo/1.4.2 requests/2.21.0 setuptools/40.6.3 requests-toolbelt/0.8.0 tqdm/4.28.1 CPython/2.7.5

File hashes

Hashes for zfs_tools_zbackup-0.5.1-py2-none-any.whl
Algorithm Hash digest
SHA256 bfcb130cfcfbe4bc744feb074b5ea65dfc8f14f5cf2d367a114fd3b8a92b17ff
MD5 4a0f8ef895a077f9f72fc9308161477c
BLAKE2b-256 a8d2dc5350a1635ad5fe48df64e8ef6f44adfa0567ff6e49c210790fc4793628

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page