LeVJEPA: Efficient & Scalable Video Pretraining without the Heuristics

TL;DR - LeVJEPA is a self-supervised video pretraining method that uses a single encoder, projector, and collapse-free regularization instead of asymmetric target encoders or pixel reconstruction. It matches or exceeds V-JEPA 2 with 5.6–20.8× less pretraining compute, making video more practical for general-purpose visual representation learning.

• Uses global/local view invariance with SIGReg, which provides a provable guarantee against representation collapse.
• Uniform random token dropping both reduces the encoder’s compute load and improves downstream accuracy.
• At matched total FLOPs, LeVJEPA beats the strongest reported video baseline by 7.6 ImageNet-1K points while remaining competitive on motion-centric tasks.
• Block-causal attention preserves accuracy and builds temporal ordering directly into the encoder.