Beyond FVD: Enhanced Evaluation Metrics for Video Generation Quality

Installation

Install our package from PyPI

pip install --upgrade videojedi

Tutorial

Our metric has a simple setup process. To compute V-JEPA features and the JEDi metric, follow these simple steps:

• Pixel values are in the range [0, 1]
• Input dimension order is T, 3, H, W (frames, channels, height, width)

from videojedi import JEDiMetric
jedi = JEDiMetric(feature_path=..., model_dir=...)
jedi.load_features(loaderA, loaderB, num_samples=...)
print(f"JEDi Metric: {jedi.compute_metric()}")

If V-JEPA features are already precomputed, simply load them and compute the JEDi metric.

from videojedi import JEDiMetric
import numpy as np
jedi = JEDiMetric()
jedi.train_features = np.random.rand(5000, 1280)
jedi.test_features = np.random.rand(5000, 1280)
print(f"JEDi Metric: {jedi.compute_metric()}") 

Follow our interactive tutorial notebook for a detailed walkthrough: Tutorial.

Citation

@misc{luo2024jedi,
        title={Beyond FVD: Enhanced Evaluation Metrics for Video Generation Quality}, 
        author={Ge Ya Luo and Gian Favero and Zhi Hao Luo and Alexia Jolicoeur-Martineau and Christopher Pal},
        year={2024},
        eprint={2410.05203},
        archivePrefix={arXiv},
        primaryClass={cs.CV},
        url={https://arxiv.org/abs/2410.05203}, 
  }