hydrusvideodeduplicator 0.4.1

Video deduplicator utility for Hydrus Network

Hydrus Video Deduplicator

Hydrus Video Deduplicator finds potential duplicate videos through the Hydrus API

---

How It Works:

The deduplicator works by comparing videos similarity by their perceptual hash.

Potential duplicates can be processed through the Hydrus duplicates processing page just like images.

You can choose to process only a subset of videos with --query using Hydrus tags, e.g. --query="character:edward" will only process videos with the tag character:edward.

For more information check out the wiki and the FAQ

---

Installation:

Dependencies:

• Python >=3.10

python3 -m pip install hydrusvideodeduplicator

---

Usage:

python3 -m hydrusvideodeduplicator --api-key="<your key>"

For full list of options see --help or the usage page.

---

TODO:

• Option to rollback and remove potential duplicates
• OR predicates for --query
• Parallelize hashing and duplicate search
• Automatically generate access key with Hydrus API
• Docker container
• Upload Docker container to Docker Hub (GitHub Action)
• Pure Python port of vpdq
• Windows compatibility without WSL or Docker

---

Contact:

Create an issue on GitHub for any problems/concerns. Provide as much detail as possible in your issue.

Message @applenanner on the Hydrus Discord for other general questions/concerns

---

Attribution:

Hydrus Network (DWTFYWTPL)

Hydrus API Library (GNU AGPLv3) by cryzed

pdq (BSD) by Meta

vpdq (BSD) by Meta

Big Buck Bunny, Sintel (CC BY 3.0) clips by Blender Foundation