Skip to main content

difPy Python Duplicate Image Finder - searches for duplicate or similar images within folders.

Project description

Duplicate Image Finder (difPy)

Tired of going through all images in a folder and comparing them manually to check if they are duplicates?

The Duplicate Image Finder (difPy) Python package automates this task for you!

Read more on how the algorithm of difPy works in my Medium article Finding Duplicate Images with Python.

For a detailed usage guide, please view the official difPy Usage Documentation.


Description

DifPy searches for images in one or two different folders, compares the images it found and checks whether these are duplicates. It then outputs the image files classified as duplicates and the filenames of the duplicate images having the lower resolution, so you know which of the duplicate images are safe to be deleted. You can then either delete them manually, or let difPy delete them for you.

DifPy does not compare images based on their hashes. It compares them based on their tensors i. e. the image content - this allows difPy to not only search for duplicate images, but also for similar images.

Basic Usage

Use the following function to make difPy search for duplicates in one specified folder:

from difPy import dif
search = dif("C:/Path/to/Folder/")

To search for duplicates within two folders:

from difPy import dif
search = dif("C:/Path/to/Folder_A/", "C:/Path/to/Folder_B/")

Folder paths must be specified as a Python string.

Output

DifPy gives two types of output that you may use depending on your use case:

A dictionary of duplicates/similar images that were found:

search.result

> Output:
{20220824212437767808 : {"filename" : "image1.jpg",
                         "location" : "C:/Path/to/Image/image1.jpg"},
                         "duplicates" : ["C:/Path/to/Image/duplicate_image1.jpg",
                                         "C:/Path/to/Image/duplicate_image2.jpg"]},
...
}

A list of duplicates/similar images that have the lowest quality:

search.lower_quality

> Output:
["C:/Path/to/Image/duplicate_image1.jpg", 
 "C:/Path/to/Image/duplicate_image2.jpg", ...]

DifPy can also generate a dictionary with statistics on the completed process:

search.stats

> Output:
{"directory_1" : "C:/Path/to/Folder_A/",
 "directory_2" : "C:/Path/to/Folder_B/",
 "duration" : {"start_date": "2022-06-13",
               "start_time" : "14:44:19",
               "end_date" : "2022-06-13",
               "end_time" : "14:44:38",
               "seconds_elapsed" : 18.6113},
 "recursive" : True,
 "similarity_grade" : "normal",
 "similarity_mse" : 200,
 "total_files_searched" : 1032,
 "total_dupl_sim_found" : 1024,
 "total_undecodable" : 5}

Additional Parameters

DifPy has the following optional parameters:

dif(directory_A, directory_B, recursive=True, similarity="normal", px_size=50, 
    show_progress=True, show_output=False, delete=False, silent_del=False)

CLI Usage

You can make use of difPy through the CLI interface by using the following commands:

python dif.py -A "C:/Path/to/Folder_A/"

python dif.py -A "C:/Path/to/Folder_A/" -B "C:/Path/to/Folder_B/"

It supports the following arguments:

dif.py [-h] -A DIRECTORY_A [-B [DIRECTORY_B]] [-Z [OUTPUT_DIRECTORY]] 
       [-r [{True,False}]] [-s [{low,normal,high,int}]] [-px [PX_SIZE]] 
       [-p [{True,False}]] [-o [{True,False}]] 
       [-d [{True,False}]] [-D [{True,False}]]

The output of difPy is then written to files and saved in the working directory, where "xxx" is a unique timestamp:

difPy_results_xxx.json
difPy_lower_quality_xxx.txt
difPy_stats_xxx.json

For a detailed usage guide, please view the official difPy Usage Documentation.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

difPy-2.4.5.tar.gz (10.0 kB view details)

Uploaded Source

File details

Details for the file difPy-2.4.5.tar.gz.

File metadata

  • Download URL: difPy-2.4.5.tar.gz
  • Upload date:
  • Size: 10.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.7.4

File hashes

Hashes for difPy-2.4.5.tar.gz
Algorithm Hash digest
SHA256 0d7c5b0e1544df20da11e5a4b4ec46a6ec47f4d93bbc23251694fad604ce590b
MD5 7db6a500a10ce0f0b6020bb045c2aea7
BLAKE2b-256 358da38010f7628126da6f39a53907925ae54fd21e49517b9048dbff7682d082

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page