Skip to main content
Join the official 2019 Python Developers SurveyStart the survey!

Finds all duplicate files in a list of files and folders

Project description

DESCRIPTION

The script finds all duplicate files in a list of files and folders.

You give it a list of files and folders, it takes all files in those folders and compares them to one another to find which ones are the same. It doesn't care about file names, directory structure or the location of the files in that structure. It cares only about bytes - whether or not two files or more are identical in their bytes.

How to use

It works like this:

import fdups
f = fdups.fdups()
f.set(["", "", ...])
f.fdups()

You set a list of files and folders and call fdups().
The list can contain any mixture of files and folders.
If you set a new list of folders it clears the previous result.

Example

>>> import fdups
>>> f = fdups.fdups()
>>> f.set(["C:\\Python27\\Scripts", "C:\\Python27\\Lib\\site-packages\\setuptools"])
>>> f.fdups()
Collecting files
Sorting
Comparing
. . . . . . 
Info:
Total number of files compared:  162
Non duplicate files:  151
Duplicate groups:  4
Total wasted space:  518.0 kB   ( 530465 )

Do you want to see the duplicate groups?
(y for Yes, anything else for No)
y

C:\Python27\Scripts\futurize.exe
C:\Python27\Scripts\pasteurize.exe
C:\Python27\Lib\site-packages\setuptools\cli-32.exe
C:\Python27\Lib\site-packages\setuptools\cli.exe

C:\Python27\Lib\site-packages\setuptools\gui-32.exe
C:\Python27\Lib\site-packages\setuptools\gui.exe

C:\Python27\Scripts\pip.exe
C:\Python27\Scripts\pip2.7.exe
C:\Python27\Scripts\pip2.exe

C:\Python27\Scripts\easy_install-2.7.exe
C:\Python27\Scripts\easy_install.exe

Do you want to see the NON duplicate files? (151 files)
(y for Yes, anything else for No)

Project details


Release history Release notifications

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for fdups, version 0.0.1
Filename, size File type Python version Upload date Hashes
Filename, size fdups-0.0.1.tar.gz (3.4 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page