🛡️ ThreatEngine

ThreatEngine is a high-performance, 100% native Rust implementation of Meta (Facebook) ThreatExchange's PDQ (Photo/Image Hashing) and TMK (Temporal Match Kernel Video Hashing) perceptual similarity algorithms with Python bindings (PyO3).

✨ Features

• 📷 PDQ (Photo Hashing): Generates 256-bit perceptual image signatures via 2D Discrete Cosine Transform (DCT) & Median Quantization.
• 🎬 TMK (Video Hashing): Aggregates frame-level PDQF signatures across video timelines using Fourier/Cosine basis functions $\psi_m(t)$ into compact video signatures.
• ⚡ Zero-Cost Performance: Native SIMD and bitwise Hamming distance calculations in Rust (10x-50x faster than pure Python loops).
• 🛡️ Robustness: Highly resistant to resizing, compression, frame-rate changes, transcoding, and minor visual modifications.
• 🐍 Python & Rust Ready: Use directly in Rust as a crate, in Python via import threatengine, or as a command-line binary.

🚀 Installation

Rust (Cargo Crate)

Add threatengine to your Cargo.toml:

[dependencies]
threatengine = "0.1.0"

Or run:

cargo add threatengine

Python Package (PyO3)

Install locally using maturin or pip:

pip install maturin
maturin develop --release

📖 Usage Examples

🦀 Rust Example

use threatengine::{generate_pdq_hash, pdq_similarity, hash_video_file};
use image::open;

fn main() {
    // 1. Image Hashing
    let img1 = open("photo1.jpg").unwrap();
    let img2 = open("photo2.jpg").unwrap();

    let (hash1, quality1) = generate_pdq_hash(&img1);
    let (hash2, quality2) = generate_pdq_hash(&img2);

    println!("Image 1 PDQ Hash: {} (Quality: {}/100)", hash1.to_hex(), quality1);
    println!("Image 2 PDQ Hash: {} (Quality: {}/100)", hash2.to_hex(), quality2);

    // Compute Hamming distance & match verdict (threshold <= 31)
    let (distance, similarity) = pdq_similarity(&hash1, &hash2);
    println!("Hamming Distance: {} bits", distance);
    println!("Similarity Score: {:.2}%", similarity * 100.0);
    println!("Verdict: {}", if hash1.is_match(&hash2, 31) { "MATCH" } else { "NO MATCH" });

    // 2. Video Hashing & Comparison
    let sig1 = hash_video_file("video1.mp4", 2.0).unwrap();
    let sig2 = hash_video_file("video2.mp4", 2.0).unwrap();

    let video_score = sig1.match_score(&sig2);
    println!("Video Similarity Score: {:.4}", video_score);
    println!("Verdict: {}", if sig1.is_match(&sig2, 0.75) { "MATCH" } else { "NO MATCH" });
}

🐍 Python Example

import threatengine

# 1. Image Hashing & Distance
hash1, quality1 = threatengine.pdq_hash_file("photo1.jpg")
hash2, quality2 = threatengine.pdq_hash_file("photo2.jpg")

distance, similarity = threatengine.pdq_similarity(hash1, hash2)
print(f"Hamming Distance: {distance} bits")
print(f"Similarity: {similarity * 100:.2f}%")
print(f"Is Match: {distance <= 31}")

# 2. Video Comparison
score = threatengine.tmk_compare_videos("video1.mp4", "video2.mp4", fps=2.0)
print(f"Video Match Score: {score:.4f}")
print(f"Is Match: {score >= 0.75}")

💻 Command-Line Interface (CLI)

# Generate image hash & quality rating
cargo run -- pdq-hash photo.jpg

# Compare two images
cargo run -- pdq-compare photo1.jpg photo2.jpg --threshold 31

# Compare two videos
cargo run -- tmk-compare video1.mp4 video2.mp4 --threshold 0.75

📊 Invariance & Robustness Matrix

🤝 Contributing

Contributions, issues, and feature requests are welcome! Feel free to check the issues page.

👨‍💻 Author

Crafted with ❤️ by Shakib Ahmed (@expertskb)

📜 License

Distributed under the MIT License. See LICENSE for more details.