🛡️ ThreatEngine

ThreatEngine is a high-performance, 100% native Rust implementation of all 4 Meta (Facebook) ThreatExchange core components: PDQ (Image), TMK (Video), vPDQ (Video PDQ), and HMA (Hasher-Matcher-Actioner) with Python bindings (PyO3).

✨ Full Component Coverage

• 📷 PDQ (Photo Hashing): 256-bit perceptual image signatures via 2D Discrete Cosine Transform (DCT) & Median Quantization.
• 🎬 TMK+PDQF (TMK Video Hashing): Aggregates frame-level PDQF signatures across video timelines using Fourier/Cosine basis functions $\psi_m(t)$.
• 📽️ vPDQ (Video PDQ): Frame-overlap video similarity algorithm for sub-clip and video segment matching based on shared similar frames.
• 🛡️ HMA (Hasher-Matcher-Actioner Engine): In-memory high-speed indexing, lookup, and action trigger platform (BLOCK, REVIEW, ALLOW) for Trust & Safety workflows.
• ⚡ Zero-Cost Performance: Native SIMD and bitwise Hamming distance calculations in Rust (10x-50x faster than pure Python loops).

🚀 Installation

Rust (Cargo Crate)

Add threatengine to your Cargo.toml:

[dependencies]
threatengine = "0.1.0"

Or run:

cargo add threatengine

Python Package (PyO3)

pip install threatengine

📖 Usage Examples

🦀 Rust Example

use threatengine::{
    generate_pdq_hash, pdq_similarity, hash_video_file,
    compute_vpdq_similarity, HmaEngine, MatchAction,
};
use image::open;

fn main() {
    // 1. PDQ Image Hashing
    let img1 = open("photo1.jpg").unwrap();
    let img2 = open("photo2.jpg").unwrap();

    let (hash1, q1) = generate_pdq_hash(&img1);
    let (hash2, q2) = generate_pdq_hash(&img2);

    let (distance, similarity) = pdq_similarity(&hash1, &hash2);
    println!("Hamming Distance: {} bits ({:.2}% similarity)", distance, similarity * 100.0);

    // 2. TMK Video Hashing
    let sig1 = hash_video_file("video1.mp4", 2.0).unwrap();
    let sig2 = hash_video_file("video2.mp4", 2.0).unwrap();
    println!("TMK Score: {:.4}", sig1.match_score(&sig2));

    // 3. HMA Engine Indexing & Action Triggering
    let mut hma = HmaEngine::new();
    hma.add_record("media_001", "known_bad_image", hash1, MatchAction::Block);

    let matches = hma.match_hash(&hash1, 31);
    println!("HMA Matches: {}, Action: {:?}", matches.len(), matches[0].action);
}

🐍 Python Example

import threatengine

# 1. Image Hashing & Distance
hash1, quality1 = threatengine.pdq_hash_file("photo1.jpg")
hash2, quality2 = threatengine.pdq_hash_file("photo2.jpg")

distance, similarity = threatengine.pdq_similarity(hash1, hash2)
print(f"Hamming Distance: {distance} bits (Match: {distance <= 31})")

# 2. TMK Video Comparison
score = threatengine.tmk_compare_videos("video1.mp4", "video2.mp4", fps=2.0)
print(f"TMK Video Match Score: {score:.4f}")

# 3. vPDQ Frame-Overlap Comparison
vpdq_score = threatengine.vpdq_match_hashes([hash1], [hash2], threshold=31)
print(f"vPDQ Similarity Score: {vpdq_score:.4f}")

👨‍💻 Author

Crafted with ❤️ by Shakib Ahmed (@expertskb)

📜 License

Distributed under the MIT License. See LICENSE for more details.