mail-deduplicate

What is Mail Deduplicate?

Provides the mdedup CLI, an utility to deduplicate mails from a set of boxes.

Mail Deduplicate

Features

Duplicate detection based on cherry-picked and normalized mail headers.
Fetch mails from multiple sources.
Reads and writes to mbox, maildir, babyl, mh, mmdf and eml formats.
Deduplication strategies based on size, timestamp, file path or random choice, chainable as fallbacks.
Copy, move, delete or hardlink the resulting set of duplicates.
Dry-run mode.
Protection against false-positives with safety checks on size and content differences.
Optional cache reusing the hashes computed by previous runs.
Parallel hashing and selection over several processes.
Supports macOS, Linux and Windows.
Standalone executables for Linux, macOS and Windows.
Shell auto-completion for Bash, Zsh and Fish.

Installation

All installation methods are available in the documentation. Below are the most popular ones:

Try it now

uv is the fastest way to run mdedup on any platform, thanks to its uvx command:

$ uvx --from mail-deduplicate -- mdedup

macOS

mdedup is part of the official Homebrew default tap, so you can install it with:

$ brew install mail-deduplicate

Executables

Standalone binaries of mdedup's latest version are available as direct downloads for several platforms and architectures:

Platform	`arm64`	`x86_64`
Linux	Download `mdedup-linux-arm64.bin`	Download `mdedup-linux-x64.bin`
macOS	Download `mdedup-macos-arm64.bin`	Download `mdedup-macos-x64.bin`
Windows	Download `mdedup-windows-arm64.exe`	Download `mdedup-windows-x64.exe`

Quickstart

Duplicate mails pile up whenever boxes get copied around: backups of the same account taken at different times, per-folder exports where a mail tagged with several labels lands in several boxes, archives consolidated from multiple clients, or IMAP synchronizations gone wrong.

mdedup cleans this up. Point it at your boxes: it groups copies of the same mail by hashing a curated set of headers, applies a --strategy to pick which copy to keep in each group, then performs an --action on the result. The default action is the safest one: sources are only read, and the deduplicated selection is written to a brand new box.

So to merge two overlapping archives into a single clean box:

$ mdedup --strategy select-one --export merged.mbox archive-2024.mbox archive-2025.mbox

Mails found in both archives are copied once into merged.mbox, along with all the mails that were unique to each source.

To remove duplicates in place instead, switch to a destructive action, and rehearse with --dry-run:

$ mdedup --strategy select-one --action delete-discarded --dry-run ~/Maildir

The hands-on tutorial builds a small playground of duplicated mails, then walks through strategies, actions and safeguards on it. The design page explains how duplicates are detected.

[!NOTE] Memory does not grow with the size of your boxes: each mail is reduced to a lightweight stub as soon as it is hashed, so a 215 MB maildir of 1,500 mails peaks at 46 MB. It does grow with their number, at roughly 1 KB of resident memory each, so a large enough collection can still exhaust your machine.

Hashing is where a run spends most of its time, and --cache reuses it between runs. The performance page covers both, and how to manage that cache.

You can support this project with pull requests, business support 🤝 and sponsorship 🫶.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

mail_deduplicate-9.3.1.tar.gz (67.6 kB view details)

Uploaded Aug 28, 2026 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

mail_deduplicate-9.3.1-py3-none-any.whl (68.2 kB view details)

Uploaded Aug 28, 2026 Python 3

File details

Details for the file mail_deduplicate-9.3.1.tar.gz.

File metadata

Download URL: mail_deduplicate-9.3.1.tar.gz
Upload date: Aug 28, 2026
Size: 67.6 kB
Tags: Source
Uploaded using Trusted Publishing? Yes
Uploaded via: uv/0.12.4 {"installer":{"name":"uv","version":"0.12.4","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for mail_deduplicate-9.3.1.tar.gz
Algorithm	Hash digest
SHA256	`4e7968e0caef32635e9ed0198e795928c8728e8f6055064023a47fe29c883f59`
MD5	`a4df1a511d749e86dbc34a42fdad05e3`
BLAKE2b-256	`936f74c987273c00163db2ea828f6e7a2e545605bce9608cf7d25ead5a1a0db0`

See more details on using hashes here.

Provenance

The following attestation bundles were made for mail_deduplicate-9.3.1.tar.gz:

Publisher: release.yaml on kdeldycke/mail-deduplicate

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: mail_deduplicate-9.3.1.tar.gz
- Subject digest: 4e7968e0caef32635e9ed0198e795928c8728e8f6055064023a47fe29c883f59
- Sigstore transparency entry: 2626066830
- Sigstore integration time: Aug 28, 2026
Source repository:
- Permalink: kdeldycke/mail-deduplicate@fc90508ec0425c6da492ffe5918fbf3e4e9d7a48
- Branch / Tag: refs/heads/main
- Owner: https://github.com/kdeldycke
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: release.yaml@fc90508ec0425c6da492ffe5918fbf3e4e9d7a48
- Trigger Event: push