xclean

File de-duplication and archiving utility.

Scan files in main to record main files to compare against.

Scan files in target to find files that are duplicates of those in main.

By default it just lists the duplicates.

Removing duplicates

Specify --remove-dups to actually remove the duplicate files.

Archiving duplicates

Specify --archive-dup-path to archive the duplicate files to another location.

Archiving new files

Specify --archive-new-path to archive any new files to another location.

Trashing duplicates

Specify --trash-dups to move the duplicate files to the trash.

Reset the database

Reset the database content of main files by specifying --clean-db.

Compare XMP files as well

To compare the file and associated XMP file specify the --check-xmp option as well.

Compare AAE files as well

To compare the file and associated AAE file specify the --check-aae option as well.

Scanning specific file types

To scan files of specific types use the --include-extensions and --exclude-extensions options.

Prompt before changing

To answer a prompt before changing a file specify --prompt

Installation

Using pypi

pip install xclean

Using pipx

pipx install xclean

Usage

usage: xclean [-h] [-m MAIN] [-t TARGET] [--store-path STORE_PATH] [--archive-dup-path ARCHIVE_DUP_PATH] [--archive-new-path ARCHIVE_NEW_PATH]
   	       [--import-new-path IMPORT_NEW_PATH] [-i [INCLUDE_EXTENSIONS ...]] [-x [EXCLUDE_EXTENSIONS ...]] [--exclude-dirs [EXCLUDE_DIRS ...]] [--unprotect]
   	       [--remove-dups] [--trash-dups] [--clean-db] [--check-xmp] [--check-aae] [-p] [--report-new] [--report-dup] [--report-summary] [--copy-not-move]
   	       [--ignore-existing] [--ignore-xattr] [--min-size MIN_SIZE]

File de-duplication and archiving utility v0.1.1

options:
-h, --help            show this help message and exit
-m MAIN, --main MAIN  Directory where main files reside
-t TARGET, --target TARGET
                    	Directory where duplicate files may reside
--store-path STORE_PATH
                    	Directory where the database is stored
--archive-dup-path ARCHIVE_DUP_PATH
                    	Archive duplicates to directory
--archive-new-path ARCHIVE_NEW_PATH
                    	Archive new files to directory
--import-new-path IMPORT_NEW_PATH
                    	Directory where to import new main files into
-i [INCLUDE_EXTENSIONS ...], --include-extensions [INCLUDE_EXTENSIONS ...]
                    	Include Extensions
-x [EXCLUDE_EXTENSIONS ...], --exclude-extensions [EXCLUDE_EXTENSIONS ...]
                    	Exclude Extensions
--exclude-dirs [EXCLUDE_DIRS ...]
                    	Exclude directories
--unprotect           Unprotect main files
--remove-dups         Remove duplicate files
--trash-dups          Trash duplicate files
--clean-db            Clean database
--check-xmp           Include xmp files when checking for duplicates
--check-aae           Include aae files when checking for duplicates
-p, --prompt          Prompt before making changes
--report-new          Report new files
--report-dup          Report duplicate files
--report-summary      Report summary of changes
--copy-not-move       Copy archive and import instead of moving
--ignore-existing     Ignore existing archive files
--ignore-xattr        Ignore Apple extended attribute files
--min-size MIN_SIZE   Minimum size to clean (default 100)

Import will archive files and import them into the database. Archive and Import always copy the associated .XMP and .AAE files if they exist.