Skip to main content

Fastdup Tool

Copyright (C) 2024 by Dr. Amir Alush and Dr. Danny Bickson.

fastdup is a tool for gaining insights from a large image/video collection. It can find anomalies, duplicate and near duplicate images/videos, clusters of similarity, learn the normal behavior and temporal interactions between images/videos. It can be used for smart subsampling of a higher quality dataset, outlier removal, novelty detection of new information to be sent for tagging.

fastdup is:

  • Unsupervised: fits any dataset
  • Scalable : handles 400M images on a single machine
  • Efficient: works on CPU only
  • Low Cost: can process 12M images on a $1 cloud machine budget

Non Commercial License

Github Project Page

System Requirements

Supported Platforms:

  • Linux
  • macOS

Windows Support: Windows is not directly supported. However, Windows users can use fastdup via Windows Subsystem for Linux 2 (WSL2) by installing Ubuntu and following the Linux installation instructions.

Metadata

Release files for fastdup 2.30

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distributions (wheels)

Table of built distributions (wheels) for fastdup 2.30
File Actions
fastdup-2.30-cp311-cp311-manylinux_2_31_x86_64.whl CPython 3.11 CPython 3.11 Linux glibc 2.31+ x86-64 Details for fastdup-2.30-cp311-cp311-manylinux_2_31_x86_64.whl
fastdup-2.30-cp311-cp311-macosx_11_0_arm64.whl CPython 3.11 CPython 3.11 macOS 11.0+ ARM64 Details for fastdup-2.30-cp311-cp311-macosx_11_0_arm64.whl
fastdup-2.30-cp310-cp310-manylinux_2_31_x86_64.whl CPython 3.10 CPython 3.10 Linux glibc 2.31+ x86-64 Details for fastdup-2.30-cp310-cp310-manylinux_2_31_x86_64.whl
fastdup-2.30-cp310-cp310-macosx_11_0_arm64.whl CPython 3.10 CPython 3.10 macOS 11.0+ ARM64 Details for fastdup-2.30-cp310-cp310-macosx_11_0_arm64.whl
fastdup-2.30-cp39-cp39-manylinux_2_31_x86_64.whl CPython 3.9 CPython 3.9 Linux glibc 2.31+ x86-64 Details for fastdup-2.30-cp39-cp39-manylinux_2_31_x86_64.whl
fastdup-2.30-cp39-cp39-macosx_11_0_arm64.whl CPython 3.9 CPython 3.9 macOS 11.0+ ARM64 Details for fastdup-2.30-cp39-cp39-macosx_11_0_arm64.whl

Total release size: 428.2 MB

Release files / fastdup-2.30-cp311-cp311-manylinux_2_31_x86_64.whl

Download URL fastdup-2.30-cp311-cp311-manylinux_2_31_x86_64.whl
Size 93.1 MB
Tags CPython 3.11 Linux glibc 2.31+ x86-64
SHA-256 checksum
How to use checksums
0ad06d03c77989bf6cd4fec227724d7c6d6f4dd037e2b9f685899684c19aa2d1
BLAKE2b-256 checksum
How to use checksums
7fde70adb663e874aaf93806ded057467040f3e43a950b56ada9e29a9147d3ec
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.1.1 CPython/3.12.5

Release files / fastdup-2.30-cp311-cp311-macosx_11_0_arm64.whl

Download URL fastdup-2.30-cp311-cp311-macosx_11_0_arm64.whl
Size 50.7 MB
Tags CPython 3.11 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
db5dae6d9225d7ae4c3292eed68943da7e1a4195ea611d164ae7bc53b8c40e17
BLAKE2b-256 checksum
How to use checksums
499d15cb6c8302a104166b603b40865e3fc961cbdde7e007aeb9e0cc7c0219e7
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.8.16

Release files / fastdup-2.30-cp310-cp310-manylinux_2_31_x86_64.whl

Download URL fastdup-2.30-cp310-cp310-manylinux_2_31_x86_64.whl
Size 92.3 MB
Tags CPython 3.10 Linux glibc 2.31+ x86-64
SHA-256 checksum
How to use checksums
959e339ef598b0908f022cb229e1fd5dfb77937f32b1f175584e2e8f8e3f3f6e
BLAKE2b-256 checksum
How to use checksums
d127fbd6b258eeeec3b7f810d29208dc03c3ab8abb2291674eec94c51472ad70
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.1.1 CPython/3.12.5

Release files / fastdup-2.30-cp310-cp310-macosx_11_0_arm64.whl

Download URL fastdup-2.30-cp310-cp310-macosx_11_0_arm64.whl
Size 49.9 MB
Tags CPython 3.10 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
b1fae0f5df105f50c796fab7e1e6f4acef9bcaf742f1177bcb10a049e6b2048f
BLAKE2b-256 checksum
How to use checksums
fce500dbfb2b8a8fd44a2b74b9f4ba0898b2611c302ff656e5a4dd63af7cc875
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.8.16

Release files / fastdup-2.30-cp39-cp39-manylinux_2_31_x86_64.whl

Download URL fastdup-2.30-cp39-cp39-manylinux_2_31_x86_64.whl
Size 92.3 MB
Tags CPython 3.9 Linux glibc 2.31+ x86-64
SHA-256 checksum
How to use checksums
c0774ef693dfd52f21046a077748b787c8a751ee40ddca08c5347b9c55bede34
BLAKE2b-256 checksum
How to use checksums
7fa99b2e1b9229fa17b3e93642e90013afe909c151141ffa3ecd386b26978959
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.1.1 CPython/3.12.5

Release files / fastdup-2.30-cp39-cp39-macosx_11_0_arm64.whl

Download URL fastdup-2.30-cp39-cp39-macosx_11_0_arm64.whl
Size 49.9 MB
Tags CPython 3.9 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
8ac9e4642c8f802c472f856d9ff9822fe9133ccd3331f73a286477c221fa0e44
BLAKE2b-256 checksum
How to use checksums
f3ae3dec898f23b412e5b5f979b25a3d77caf733dab7451fdcbd4a402d83baa8
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.8.16

Security for fastdup 2.30

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page