Skip to main content

Incremental backup

Project description

A Python class for file comparison and new file backup.

Author: Yu Sun at University of Sydney




When it comes to backing up a large amout of data, it is often preferable to only copy the modified and unique files, rather than simply coping the whole directory. The AmendmentBackUp (ABU) class provides a simple interface to do that. No dependencies are required apart from the Python 3 standard library.


Say we have two folders, a source folder dir1 which you have your most recent files and a reference folder dir2 which holds some of your previous backup. What the ABU does is to compare all files in dir1 with those in dir2, and copy the files to a third destination folder dst. If you simply want to add the files to the original back, you can set dst to dir2.

A quick example

abu = AmendmentBackUp(dir1=r"demo/dir1",


Say you have the dir1 and dir2 with the following tree structures:

        dir1 (source, recently updated)
        |   file1.txt
        |   file2.txt (modified)
        |   file3.txt (unique)
        |       file4.txt
        |       file5.txt
        |       file6.txt (modified)
        \---subfolder3 (unique)

        dir2 (reference, previous backup)
        |   file1.txt
        |   file2.txt
        |   file7.txt
        |       file4.txt

You can inititate an `ABU` object by calling

abu = AmendmentBackUp(r’demo/dir1’, r’demo/dir2’, r’demo/dst’) ``By the way, thecreateDemo()` will create the demo file structures shown above.

After initiation, what ABU will do: * Comparison ( by walking through all files and folders in dir1 and check the existence of the corresponding counter in dir2. * If no, then add to the copy list. For folders, this will be the entire folder. * If yes, compare the two folders (shallow comparison using the time stamp and the file size) for files; * If the comparison returns False, add to the copy list; * Otherwise, continue to the next one. * Copy (abu.backup()) the files and folders in the copy list. * Folders will be copied first. If the parent folder has been copied, any child folder will be skipped; * Files will copied next. If the file falls under any folder from the previous step, it will be skipped.

  • The meta data of the backup process will be store in a folder called _abu with a time stamp (year-month-day-hour-minute-second) in the dst folder.


The ABU is best suited for when the source folder dir1 is a natural growth of the reference folder dir2. What I mean by natural growth is that there should not be too much renaming or move of the subfolders from dir2 to dir1. If that’s the case, using a version control system is probably a better option.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

amendment-back-up-0.0.5.tar.gz (5.2 kB view hashes)

Uploaded source

Built Distribution

amendment_back_up-0.0.5-py3-none-any.whl (8.2 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page