Skip to main content

📧 CLI to deduplicate mails from mail boxes

Project description

Mail Deduplicate

Last release Python versions Downloads Unittests status Coverage status Documentation status DOI

What is Mail Deduplicate?

Provides the mdedup CLI, an utility to deduplicate mails from a set of boxes.

Mail Deduplicate

Features

  • Duplicate detection based on cherry-picked and normalized mail headers.
  • Fetch mails from multiple sources.
  • Reads and writes to mbox, maildir, babyl, mh and mmdf formats.
  • Deduplication strategies based on size, content, timestamp, file path or random choice.
  • Copy, move or delete the resulting set of duplicates.
  • Dry-run mode.
  • Protection against false-positives with safety checks on size and content differences.
  • Supports macOS, Linux and Windows.
  • Standalone executables for Linux, macOS and Windows.
  • Shell auto-completion for Bash, Zsh and Fish.

Installation

All installation methods are available in the documentation. Below are the most popular ones:

Try it now

uv is the fastest way to run mdedup on any platform, thanks to its uvx command:

$ uvx --from mail-deduplicate -- mdedup

macOS

mdedup is part of the official Homebrew default tap, so you can install it with:

$ brew install mail-deduplicate

Executables

Standalone binaries of mdedup's latest version are available as direct downloads for several platforms and architectures:

Platform arm64 x86_64
Linux Download mdedup-linux-arm64.bin Download mdedup-linux-x64.bin
macOS Download mdedup-macos-arm64.bin Download mdedup-macos-x64.bin
Windows Download mdedup-windows-arm64.exe Download mdedup-windows-x64.exe

Quickstart

[!WARNING] Performance and memory usage: mdedup implementation is quite naive and everything resides in memory.

If this is good enough for a volume of a couple of gigabytes, the more emails mdedup try to parse, the closer you'll reach the memory limits of your machine. In which case mdedup will exit abruptly, zapped by the OOM killer of your OS. Of course your mileage may vary depending on your hardware.

You can influence implementation of this feature with pull requests, purchasing business support 🤝 and sponsorship 🫶.

Example

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

mail_deduplicate-8.1.2.tar.gz (37.1 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

mail_deduplicate-8.1.2-py3-none-any.whl (32.7 kB view details)

Uploaded Python 3

File details

Details for the file mail_deduplicate-8.1.2.tar.gz.

File metadata

  • Download URL: mail_deduplicate-8.1.2.tar.gz
  • Upload date:
  • Size: 37.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.9.14 {"installer":{"name":"uv","version":"0.9.14","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for mail_deduplicate-8.1.2.tar.gz
Algorithm Hash digest
SHA256 30ed948490f35f47da27829f239c20068a936010d78e198da44b69f1d65c3d1e
MD5 09d42f499421986eac32ecaa358db916
BLAKE2b-256 0c0c0e83bc1a549394b6c372caa8e6e5859020d23f470362d02276f1d2d56645

See more details on using hashes here.

File details

Details for the file mail_deduplicate-8.1.2-py3-none-any.whl.

File metadata

  • Download URL: mail_deduplicate-8.1.2-py3-none-any.whl
  • Upload date:
  • Size: 32.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.9.14 {"installer":{"name":"uv","version":"0.9.14","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for mail_deduplicate-8.1.2-py3-none-any.whl
Algorithm Hash digest
SHA256 35c459f3b80a21e86d78933fc9a68ef38ebe41e99afdb62f54cd4446c54f2599
MD5 6b1985fdff85f9bdaacf6a9fd064b666
BLAKE2b-256 f8a4bdfdcf52bd8a1ffcd285408aad18e511b5b8693e915d2dc8bff0f1c84146

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page