ghost-remover 0.3.1

AI Video Inpainting Engine - remove watermarks and objects from video using SAM 2 + ProPainter

Ghost Remover

AI-powered video inpainting engine. Remove watermarks, logos, and objects from video using SAM 2 + ProPainter.

Features

• SAM 2 tracking - Segment anything with text, box, or point prompts via Grounding DINO + SAM 2
• ProPainter inpainting - Temporal-aware video inpainting with optical flow completion
• Two-pass architecture - Pass 1: tracking + mask generation. Pass 2: inpainting + streaming encode
• Chunked processing - Handles arbitrarily long videos with overlap blending
• MPS optimized - Native Apple Silicon support with automatic CPU fallback for unsupported ops
• Checkpoint resume - Interrupt and resume without losing progress
• Batch processing - Process multiple videos in one command
• Quality metrics - Temporal consistency, pixel integrity (PSNR), and style (LPIPS) checks

v0.3 Performance Upgrades

Optimization	Effect
Eliminate redundant ffprobe	frames_to_numpy accepts video_info param, skips re-probing
Single decode in Pass 1	Reuses extracted JPEGs for re-detection fallback instead of second FFmpeg decode
Float16 mixed precision	torch.autocast("mps", float16) for ProPainter inference — ~1.5x faster, half memory
Chunk prefetch	Background thread decodes next chunk while current chunk is being inpainted
Vectorized blending	Overlap blending uses numpy batch ops instead of per-frame Python loop
OOM auto-retry	On memory error: split chunk in half → retry → downscale fallback → original frames
FFmpeg timeout	All subprocess calls have timeout protection (120-300s)
Frame count validation	Warns when decoded frame count doesn't match expected

Installation

# Create conda environment
conda env create -f environment.yml
conda activate ghost-remover

# Clone ProPainter into third_party/
git clone https://github.com/sczhou/ProPainter.git third_party/ProPainter
bash setup_propainter.sh

# Download SAM 2 checkpoints
mkdir -p checkpoints
# See https://github.com/facebookresearch/segment-anything-2#download-checkpoints

Usage

CLI

# Basic - remove watermark by text prompt
ghost-remover remove --video input.mp4 --target "watermark"

# With bounding box
ghost-remover remove --video input.mp4 --target "logo" --box 100,50,300,150

# Dry-run (preview mask overlay only)
ghost-remover remove --video input.mp4 --target "watermark" --dry-run

# Batch processing
ghost-remover remove --video a.mp4 b.mp4 c.mp4 --target "watermark"

# Hardware-accelerated encoding (VideoToolbox)
ghost-remover remove --video input.mp4 --target "watermark" --hw-accel

# Adjust sensitivity (0.1=tight, 1.0=aggressive)
ghost-remover remove --video input.mp4 --target "watermark" --sensitivity 0.7

Python API

from main import GhostRemoverPipeline

config = {
    "video_path": "input.mp4",
    "target": "watermark",
    "sensitivity": 0.5,
    "sam_model_size": "large",
}

pipeline = GhostRemoverPipeline(config)
output = pipeline.run()
pipeline.cleanup()

Architecture

ghost-remover/
  core/
    ffmpeg_handler.py   # Video decode/encode, streaming pipe I/O
    tracker_sam3.py     # SAM 2 segmentation + video tracking
    inpainter_pp.py     # ProPainter inpainting engine
  utils/
    gpu_manager.py      # MPS memory monitoring + chunk size estimation
    video_chunker.py    # Chunk planning + overlap blending
    checkpoint.py       # Interrupt recovery
    quality_metrics.py  # Temporal/pixel/style quality checks
  api/
    skill_interface.py  # CLI + JSON API
  main.py               # Two-pass pipeline orchestrator

Pipeline Flow

Input Video
  │
  ├─ Pass 1: SAM 2 Tracking (5-50%)
  │    ├─ First frame segmentation (text/box/point prompt)
  │    ├─ Per-chunk mask propagation
  │    ├─ Drift detection + re-detection
  │    └─ Packed mask storage (.npz)
  │
  ├─ Pass 2: ProPainter Inpainting (50-95%)
  │    ├─ RAFT optical flow (fp16)
  │    ├─ Bidirectional flow completion
  │    ├─ Image propagation
  │    ├─ Transformer inpainting (sliding window)
  │    ├─ Overlap blending (vectorized)
  │    └─ Streaming FFmpeg encode
  │
  └─ Finalize (95-100%)
       ├─ Audio merge
       └─ Quality check

Tests

# Run all tests (134 tests)
python3 -m pytest tests/ghost-remover/ -v

Requirements

• Python >= 3.11
• PyTorch >= 2.2.0 (MPS support)
• FFmpeg
• SAM 2 (segment-anything-2)
• ProPainter (third_party/)
• Grounding DINO (via HuggingFace transformers, optional for text prompts)

Version

v0.3.0 — 134 tests passing

License

MIT