rsync-time-machine.py is a wrapper around rsync to make it easier to create and maintain Time Machine style backups.
Project description
Rsync Time Machine 🕰️💾
Introducing rsync-time-machine.py
- a Python port of the rsync-time-backup
script, offering Time Machine-style backups using rsync. It creates incremental backups of files and directories to the destination of your choice. The backups are structured in a way that makes it easy to recover any file at any point in time. 🚀
It works on Linux, macOS, and Windows (via WSL or Cygwin). The main advantage over Time Machine is flexibility, as it can backup from/to any filesystem and works on any platform. You can also backup to a Truecrypt drive without any problem. 😃
rsync-time-machine.py
is fully tested, has no external dependencies (only Python ≥3.7 🐍), is fully compatible with rsync-time-backup
, offers pretty terminal output, and is fully typed! 🎉
[ToC] 📚
- :star2: Features
- :books: Usage
- :hammer_and_wrench: Installation
- :bulb: Examples
- :calendar: Backup Expiration Logic
- :page_facing_up: Exclusion File
- :lock: Built-in Lock
- :gear: Rsync Options
- :no_entry_sign: No Automatic Backup Expiration
- :arrows_counterclockwise: How to Restore
- :star: Featured on
- :heart: Support and Contributions
:star2: Features
- 📁 Each backup is in its own folder named after the current timestamp.
- 🔒 Backup to/from remote destinations over SSH.
- 🔗 Files that haven't changed from one backup to the next are hard-linked to the previous backup, saving space.
- ⚠️ Safety check - the backup will only happen if the destination has explicitly been marked as a backup destination.
- 🔄 Resume feature - if a backup has failed or was interrupted, the tool will resume from there on the next backup.
- 🚫 Exclude file - support for pattern-based exclusion via the
--exclude-from
rsync parameter. - 🧹 Automatically purge old backups based on a configurable expiration strategy.
- 🔗 "latest" symlink that points to the latest successful backup.
:books: Usage
To use rsync-time-machine.py
, you'll need to provide source and destination paths, along with any desired options:
rsync-time-machine --help
Shows the help message:
usage: rsync-time-machine [-h] [-p PORT] [-i ID_RSA] [--rsync-get-flags]
[--rsync-set-flags RSYNC_SET_FLAGS]
[--rsync-append-flags RSYNC_APPEND_FLAGS]
[--log-dir LOG_DIR] [--strategy STRATEGY]
[--no-auto-expire] [--allow-host-only]
[--exclude-from EXCLUDE_FROM] [-v]
src_folder dest_folder [exclusion_file]
A script for creating and managing time-stamped backups using rsync.
positional arguments:
src_folder Source folder for backup. Format: [USER@HOST:]SOURCE
dest_folder Destination folder for backup. Format:
[USER@HOST:]DESTINATION
exclusion_file Path to the file containing exclude patterns. Cannot
be used together with `--exclude-from`.
options:
-h, --help show this help message and exit
-p PORT, --port PORT SSH port.
-i ID_RSA, --id_rsa ID_RSA
Specify the private ssh key to use.
--rsync-get-flags Display the default rsync flags that are used for
backup. If using remote drive over SSH, --compress
will be added.
--rsync-set-flags RSYNC_SET_FLAGS
Set the rsync flags that are going to be used for
backup.
--rsync-append-flags RSYNC_APPEND_FLAGS
Append the rsync flags that are going to be used for
backup.
--log-dir LOG_DIR Set the log file directory. If this flag is set,
generated files will not be managed by the script - in
particular they will not be automatically deleted.
Default: $HOME/.rsync-time-backup
--strategy STRATEGY Set the expiration strategy. Default: "1:1 30:7
365:30" means after one day, keep one backup per day.
After 30 days, keep one backup every 7 days. After 365
days keep one backup every 30 days.
--no-auto-expire Disable automatically deleting backups when out of
space. Instead, an error is logged, and the backup is
aborted.
--allow-host-only By default, the script expects a 'USER@HOST' pattern
for specifying SSH connections. When this flag is
used, it allows for the 'HOST' pattern without a
specified user. This is useful if you want to use
configurations from the `.ssh/config` file or rely on
the current username. Note: this option will not
enforce SSH usage, it only broadens the accepted input
formats.
--exclude-from EXCLUDE_FROM
Path to the file containing exclude patterns.
Alternative to the positional `exclusion_file`. Not to
be used with `exclusion_file`.
-v, --verbose Enable verbose output. This will slow down the backup
process (in simple tests by 2x).
Please refer to the original rsync-time-backup
README for a list of options, as they have been preserved in the Python port.
:hammer_and_wrench: Installation
To install rsync-time-machine.py
, simply clone the repository:
pip install rsync-time-machine
and use it like rsync-time-machine --help
Or just copy the script to your computer:
wget https://raw.githubusercontent.com/basnijholt/rsync-time-machine.py/main/rsync_time_machine.py
and use it like ./rsync_time_machine.py --help
:bulb: Examples
- Backup the home folder to backup_drive:
./rsync_time_machine.py /home /mnt/backup_drive
- Backup with exclusion list:
./rsync_time_machine.py /home /mnt/backup_drive excluded_patterns.txt
For more examples and detailed usage instructions, please refer to the original rsync-time-backup
README.
:calendar: Backup Expiration Logic
Backup sets are automatically deleted following a simple expiration strategy defined with the --strategy
flag. The default strategy is 1:1 30:7 365:30
. Please see the original README for a detailed explanation.
:page_facing_up: Exclusion File
An optional exclude file can be provided as a third parameter, compatible with the --exclude-from
parameter of rsync.
The --exclude-from
option in rsync-time-machine.py
allows you to exclude specific files or directories from the backup process. You can provide an exclusion file containing patterns for files or directories that should be excluded.
📖🔽 Click here to expand the docs on --exclude-from
🔽📖
Here's how to use the --exclude-from
feature in rsync-time-machine.py
:
- Create a text file named
exclusion_file.txt
(or any other name you prefer) in your preferred location. - Add the exclusion patterns to the file, one pattern per line. Patterns can be literal strings, wildcards, or character ranges.
- Save the file.
To use this exclusion file while performing a backup with rsync-time-machine.py
, include it as the third positional argument in your command (or with --exclude-from exclusion_file.txt
). For example:
rsync-time-machine.py /home /mnt/backup_drive exclusion_file.txt
In this example, /home
is the source folder, /mnt/backup_drive
is the destination folder, and exclusion_file.txt
contains the exclude patterns.
Here's a sample exclusion_file.txt
:
+ /home/.fileA
- /home/.*
- /home/junk/
In this example:
+ /home/.fileA
: Include the file.fileA
from thehome
directory.- /home/.*
: Exclude all hidden files (files starting with a dot) from thehome
directory.- /home/junk/
: Exclude the entirejunk
directory from thehome
directory.
Remember that the order of patterns matters, as rsync reads the file top-down and acts on the first matching pattern it encounters.
See this tutorial for more information.
:lock: Built-in Lock
The script is designed so that only one backup operation can be active for a given directory, avoiding conflicts.
:gear: Rsync Options
To display, add, or remove rsync options, use the --rsync-get-flags
, --rsync-append-flags
, or --rsync-set-flags
options.
:no_entry_sign: No Automatic Backup Expiration
Use the --no-auto-expire
flag to disable the default behavior of purging old backups when out of space.
:arrows_counterclockwise: How to Restore
Restoring files from the backup is simple, as the script creates a backup in a regular directory. You can easily copy the files back to the original directory using a command like:
rsync -aP /path/to/last/backup/ /path/to/restore/to/
Consider using the --dry-run
option to check what exactly is going to be copied. If you want to delete files that exist in the destination but not in the backup, use the --delete
option. Be extra cautious when using this option to avoid data loss.
You can also restore files using any file explorer, including Finder on macOS or the command line.
:star: Featured on
- the Real Python podcast: Episode 158: Building Python CI With Docker & Applying for a Hacker Initiative Grant @ 00:26:28
- Y Combinator Hacker News: Python Port of 600 Line Bash Script: rsync-time-machine.py for Rsync Backups (self-posted)
- Reddit /rpython: Ported a popular (untested) 600+ Line Bash Script 📜 to Python 🐍: Introducing rsync-time-machine.py for Time Machine-Style Backups Using Rsync 🔄⏰ (self-posted)
:heart: Support and Contributions
We appreciate your feedback and contributions! If you encounter any issues or have suggestions for improvements, please file an issue on the GitHub repository. We also welcome pull requests for bug fixes or new features.
Happy backing up! 💾🕰️🎉
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file rsync-time-machine-1.3.3.tar.gz
.
File metadata
- Download URL: rsync-time-machine-1.3.3.tar.gz
- Upload date:
- Size: 21.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.0.0 CPython/3.12.2
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0d0c390f2d133ab7829e8188c592471ac4b8ab8c81bbf74d84ba8d7d816061d5 |
|
MD5 | 001af9e807da2442ce89c3426b4c8504 |
|
BLAKE2b-256 | 55cda65a8dd6407a1526fdca34f5f5776e61142843baadc105a2da3b96a7a509 |
File details
Details for the file rsync_time_machine-1.3.3-py3-none-any.whl
.
File metadata
- Download URL: rsync_time_machine-1.3.3-py3-none-any.whl
- Upload date:
- Size: 18.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.0.0 CPython/3.12.2
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4d73018163b2322651790e5bcbce47809c37f3ff44746be1d44273486cd6339f |
|
MD5 | 8bffa569f38b9d322196f408183e3834 |
|
BLAKE2b-256 | b3dfc95c2707a920a2d408b957409563bddf217348c6a2ce711ea664cb4e631f |