Duplicity 4

Duplicity 4 (duplicity4) backs directories by producing encrypted, tar-format volumes and uploading them to a remote or local storage backend. By using librsync and the rsync algorithm, Duplicity generates bandwidth- and space-efficient incremental archives that only record binary deltas of changed files. Archives are encrypted and signed using GnuPG (via GPGME) to ensure privacy, integrity, and authenticity.

Key Features

• Concurrent Backend Worker Pool: Multi-worker asynchronous volume compression, encryption, and upload pipeline (src/backend_pool.py) for significantly faster backups.
• Concurrent Restore Prefetching: Asynchronous volume prefetching for restore and verification operations.
• Hardware-Accelerated Compression: High-throughput compression using zlib-ng.
• Advanced File Handling: Native sparse file detection and efficient chunking (--sparse) alongside preserved hardlink reconstruction (--hardlinks).
• Encrypted & Signed Backups: Uses GnuPG via GPGME to encrypt and sign archives with standard OpenPGP or CMS (X.509) keys.
• Bandwidth & Space Efficient: Employs librsync to compute binary deltas for changed files in incremental backups.
• Standard Archive Format: Volumes are standard POSIX tar format archives encrypted with OpenPGP/CMS.
• Extensive Storage Backend Support: Backups can be stored locally or uploaded to cloud and remote services (SSH/SFTP, rsync, Amazon S3, Backblaze B2, Google Cloud Storage, Microsoft Azure Blob, Google Drive, Microsoft OneDrive, Dropbox, WebDAV, Swift, Mega, and more).
• Restartable Transfers: Asynchronous volume generation and transfer resume support for reliable operation over interrupted connections.
• Standardized Logging: Python standard logging integration with named severity levels (error, warning, notice, info, debug, critical) and machine-readable output options.

Quick Start Examples

1. Perform a Backup

# Backup local directory to a remote SFTP destination
duplicity /home/user/data pex+sftp://user@example.com/backup

# Force a full backup
duplicity full /home/user/data pex+sftp://user@example.com/backup

2. Restore Files

# Restore the entire latest backup to a destination folder
duplicity restore pex+sftp://user@example.com/backup /home/user/restore

# Restore a specific file from 3 days ago
duplicity restore --file-to-restore document.txt --time 3D pex+sftp://user@example.com/backup /home/user/document.txt

3. Verify and Compare Backup Integrity

# Compare remote backup contents against the local filesystem
duplicity compare pex+sftp://user@example.com/backup /home/user/data

# Verify remote backup volume checksums and integrity
duplicity verify pex+sftp://user@example.com/backup /home/user/data

4. Check Backup Status and List Files

# Show collection status (backup sets and volume chains)
duplicity collection-status pex+sftp://user@example.com/backup

# List all files currently contained in the backup archive
duplicity list-current-files pex+sftp://user@example.com/backup

System Requirements

• Python: Python >= 3.10 (tested through Python 3.15).
• C Libraries: librsync (>= 0.9.7, 2.x recommended), libgpgme (GPGME C library), zlib-ng.
• GnuPG: gpg (2.x), gpgsm, and python3-gpg (GPGME Python bindings).
• Build Tools: C compiler (gcc or clang), make, python3-dev, swig, pkg-config.

Documentation & Guides

Comprehensive guides for users, packagers, and developers are organized in the README/ directory and documentation:

• Installation Guide — Comprehensive installation instructions (pipx, apt, pip venv, source).
• Architecture & Design — Internal design, volume formats, signature chains, and encryption workflow.
• Repository & Development Guide — Source tree setup, building C extensions, and running from source.
• Testing Guide — Running tests with Docker, pytest, parallel workers, and test isolation guidelines.
• Call for Testing — Call for testing Duplicity 4.0 pre-releases, focus areas, and feedback channels.
• Logging Subsystem & Protocol — Named log levels, command-line flags, and machine-readable output protocol.
• Release Process — Release checklists, versioning guidelines, Launchpad PPAs, snaps, and PyPI packaging.
• Snap Packaging — Building and testing multi-architecture Snap packages.
• Sphinx Documentation Guide — Building HTML documentation locally and ReadTheDocs configuration.
• Backend Development Guide — Guide to implementing and testing new storage backends.
• Test Keyring Reference — Test GnuPG keys, fingerprints, and passphrases.

Help & Community

• Official Website: https://duplicity.us
• Project Repository: https://gitlab.com/AlphaPrime/duplicity4.git
• Issue Tracker: https://gitlab.com/AlphaPrime/duplicity4/-/issues
• Online Documentation: https://duplicity4.readthedocs.io/
• Mailing List: https://lists.nongnu.org/mailman/listinfo/duplicity-talk
• Translation Portal: https://crowdin.com/project/duplicity