Rikugan

Local, offline deepfake video detection tool. Upload a video, get a forensic analysis with heatmap overlays, per-region breakdowns, and a downloadable PDF report.

No cloud upload. No black box. Every signal and threshold is disclosed.

Overview

Rikugan applies three independent forensic signals to detect manipulation artifacts in video:

• Spatial (55%) — EfficientNet-V2 CNN classifies face crops as real/fake with Grad-CAM heatmaps showing which pixels influenced the decision. Per-region scoring across 6 facial zones (eyes, nose, mouth, forehead, cheeks, jawline).
• Artifact (20%) — Forensic artifact detection analyzing skin texture incongruence between face regions, boundary gradient anomalies at face-swap seams, specular highlight consistency, and spatial noise pattern uniformity.
• rPPG (30%) — Remote photoplethysmography (rPPG) pulse signal analysis via green-channel extraction and FFT-based frequency decomposition. Anomalous or absent pulse patterns suggest synthetic content.

Scores are fused into a weighted composite. No single signal is definitive — the result reflects agreement across independent detection methods.

Quick Start

Installation

pip install rikugan

# Download model weights (required)
rikugan download-weights

Running

rikugan

Opens at http://127.0.0.1:8000. Upload a video (MP4, MOV, AVI, or WEBM, max 3 minutes, 500 MB) and view the analysis.

Development (contributors)

git clone https://github.com/miiidev/rikugan.git
cd rikugan
python -m venv .venv
.venv\Scripts\activate        # Windows
# source .venv/bin/activate   # Linux/Mac
pip install -e .

# Frontend
cd frontend
npm install
npm run build
cd ..

# Or use the build helper:
python scripts/build_frontend.py

rikugan

Architecture

Upload video
     |
     v
 Frame Extraction (OpenCV, every Nth frame)
     |
     v
 Face Detection (MediaPipe FaceLandmarker, 478 landmarks)
     |
  +---> Spatial CNN (EfficientNet-V2 + Grad-CAM + per-region scoring)
      |
      +---> Artifact Detection (texture + boundaries + highlights + noise)
      |
      +---> rPPG (Pulse) Analysis (green-channel FFT + SNR scoring)
     |
     v
 Score Fusion (weighted sum -> suspicion level + explanations)
     |
     v
 PDF Report + Frontend Results Dashboard

Suspicion Levels

Project Structure

rikugan/
├── backend/
│   ├── app/                  # FastAPI application
│   │   ├── main.py           # App instance, CORS, static file serving
│   │   ├── config.py         # Paths, limits, server settings
│   │   ├── task_store.py     # In-memory task state tracking
│   │   ├── history_store.py  # JSON-file persistence for past analyses
│   │   ├── routers/          # API routes (upload, status, download, history, metrics)
│   │   └── schemas/          # Pydantic request/response models
│   ├── detector/             # Core detection pipeline
│   │   ├── extraction.py     # Frame extraction from video
│   │   ├── face_detection.py # MediaPipe face detection + cropping
│   │   ├── spatial_cnn.py    # EfficientNet-V2 inference + Grad-CAM
│   │   ├── artifact.py       # Forensic artifact detection
│   │   ├── rppg.py           # Remote photoplethysmography (rPPG) pulse analysis
│   │   ├── fusion.py         # Score fusion + explanations
│   │   ├── heatmap.py        # Bounding box overlay rendering
│   │   ├── report.py         # PDF report generation (ReportLab)
│   │   └── pipeline.py       # Pipeline orchestrator
│   └── cli.py                # Typer CLI entry point
├── frontend/
│   └── src/
│       ├── pages/            # Landing, Upload, Processing, Results, History, About, Methodology
│       ├── components/       # VideoPlayer, ConfidenceGraph, Sidebar, etc.
│       └── api/client.ts     # Axios API client + TypeScript interfaces
├── scripts/
│   ├── preprocess.py         # FaceForensics++ dataset preprocessing
│   ├── train.py              # EfficientNet-V2 training (AdamW, cosine annealing, mixed precision)
│   ├── evaluate.py           # Model evaluation with per-method breakdown
│   ├── score_distribution.py # Full pipeline evaluation on sampled videos
│   └── diagnose_pipeline.py  # Per-step pipeline debugging tool
├── weights/                  # Model weights (gitignored)
│   ├── ev2_best.pth          # Trained PyTorch checkpoint (EfficientNet-V2)
│   └── face_landmarker.task  # MediaPipe face detection model
├── data/
│   ├── raw/                  # FaceForensics++ dataset
│   ├── processed/            # Extracted face crops
│   └── splits/               # Train/test splits + evaluation results
└── pyproject.toml

Technical Details

Model

• Architecture: EfficientNet-V2-S (compound scaling, Fused-MBConv blocks)
• Input: 384x384 RGB face crops
• Output: Binary classification (real vs fake) with softmax probability
• Training: FaceForensics++ c23 quality, 80/10/10 stratified split with identity leakage prevention, AdamW optimizer with cosine annealing + warmup, label smoothing, gradient clipping, mixed precision
• Post-retrain accuracy: 99.07% on test set

Face Detection

MediaPipe FaceLandmarker with 478 facial landmarks. Faces are cropped and resized to 384x384 for CNN input. Landmarks are used for per-region scoring and artifact analysis.

Dataset

FaceForensics++ with 4 manipulation methods:

• Deepfakes — autoencoder-based face swap
• Face2Face — face reenactment
• FaceSwap — 3D morphable model face swap
• NeuralTextures — neural texture manipulation

If you use this dataset, please cite:

@inproceedings{roessler2019faceforensicspp,
	author = {Andreas R\"ossler and Davide Cozzolino and Luisa Verdoliva and Christian Riess and Justus Thies and Matthias Nie{\ss}ner},
	title = {Face{F}orensics++: Learning to Detect Manipulated Facial Images},
	booktitle= {International Conference on Computer Vision (ICCV)},
	year = {2019}
}

Limitations

• Videos longer than 3 minutes are not supported
• Only one face per frame is analyzed (primary detected face)
• The model was trained on FaceForensics++ (c23 quality) — performance may vary on other manipulation methods or compression levels
• No automated system achieves 100% accuracy — false positives and false negatives occur
• Results should be interpreted alongside other evidence and domain expertise

Tech Stack

Backend: Python, FastAPI, PyTorch, MediaPipe, OpenCV, ReportLab

Frontend: React 19, TypeScript, Tailwind CSS v4, Recharts, Axios, Vite

Dataset: FaceForensics++ (YouTube faces, 4 manipulation methods)