Duplicates
Identify duplicate files and replace them with hardlinks on any OS.
Intended to be used to reduce the storage space taken up by mutliple copies of similar backups. (E.g. regular google takeouts)
Usage
Can be run from a command line in Linux, MacOS or Windows and will recursively scan a directory, identify and optionally hardlink any duplicate files found.
WARNING: Hardlinking files means if you change any one "copy" all "copies" will change.
WARNING: If other hardlinks are present outside the directories scanned, these may no longer point to the same inode as those within the scanned directories. Consider the situation as undefined.
Command line
dupes PATH will display number of duplicate files found under PATH
dupes PATH1 PATH2 ... will display number of duplicate files found under and across PATH1 and PATH2
dupes --list PATHS... will list the full sets of duplicate files found
dupes --short PATHS... will only list sets of duplicates where there are different file names
and finally ...
dupes --link PATHS... will replace duplicate files with hard links
Python
You can also use the class DuplicateFiles to indentify and optionally link duplicates.
Additionally BufferedIOFile provides a binary file which knows its Path and offers a readchunk() method similar to the text file readline().
Up Next
- Keep original file mode after hardlinking
- Select leading inode for linking
- Improved exception handling from the command line
Please vote on any issues which are important to you.
1 maintainer
Metadata
1 maintainer
Release files for link-duplicates 1.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| link-duplicates-1.1.0.tar.gz | 7.2 kB | Without provenance | Details for link-duplicates-1.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| link_duplicates-1.1.0-py3-none-any.whl | Python 3 | none | any | Details for link_duplicates-1.1.0-py3-none-any.whl |
Total release size: 14.8 kB
Release files / link-duplicates-1.1.0.tar.gz
| Download URL | link-duplicates-1.1.0.tar.gz |
|---|---|
| Size | 7.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
9a0f9528b91dbe69e968d20f7fc6a059e311c54bbab61e50bc24a3af9ee9f7db
|
|
BLAKE2b-256 checksum How to use checksums |
e493405475e355e3cdac72bef303e1e98817353ae752cc436761e7b7dfa7cbd5
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/4.0.2 CPython/3.11.7
|
Release files / link_duplicates-1.1.0-py3-none-any.whl
| Download URL | link_duplicates-1.1.0-py3-none-any.whl |
|---|---|
| Size | 7.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
c90abad337e1d1d5925e73ba4d756a500e9887d64b10b23cf72dd8d8acbbdb85
|
|
BLAKE2b-256 checksum How to use checksums |
dbb01b8bfe932f75feb03100972c50da93d23849424320a91c0f8f90252ed7df
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/4.0.2 CPython/3.11.7
|
Security for link-duplicates 1.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
