LeVJEPA: Efficient & Scalable Video Pretraining without the Heuristics
Ranking
Observed public metrics from 1 member.
Merged summary
TL;DR - LeVJEPA is a self-supervised video pretraining method that uses a single encoder, projector, and collapse-free regularization instead of asymmetric target encoders or pixel reconstruction. It matches or exceeds V-JEPA 2 with 5.6–20.8× less pretraining compute, making video more practical for general-purpose visual representation learning.
- Uses global/local view invariance with SIGReg, which provides a provable guarantee against representation collapse.
- Uniform random token dropping both reduces the encoder’s compute load and improves downstream accuracy.
- At matched total FLOPs, LeVJEPA beats the strongest reported video baseline by 7.6 ImageNet-1K points while remaining competitive on motion-centric tasks.
- Block-causal attention preserves accuracy and builds temporal ordering directly into the encoder.
Sources (1)
LeVJEPA: Efficient & Scalable Video Pretraining without the Heuristics
TL;DR - LeVJEPA is a self-supervised video pretraining method that uses a single encoder, projector, and collapse-free regularization instead of asymmetric target encoders or pixel reconstruction. It matches or exceeds V-JEPA 2 with 5.6–20.8× less pretraining compute, making video more practical for general-purpose visual representation learning.
- Uses global/local view invariance with SIGReg, which provides a provable guarantee against representation collapse.
- Uniform random token dropping both reduces the encoder’s compute load and improves downstream accuracy.
- At matched total FLOPs, LeVJEPA beats the strongest reported video baseline by 7.6 ImageNet-1K points while remaining competitive on motion-centric tasks.
- Block-causal attention preserves accuracy and builds temporal ordering directly into the encoder.