Orthogonal JEPA: Factorized Predictive States for Latent World Models

TL;DR - Orthogonal JEPA is a latent world-modeling framework that factorizes predictive states into multiple orthogonal components rather than using one monolithic target embedding. The design aims to reduce redundant capacity, preserve less-dominant predictive structure, and support more stable forecasting, planning, and rollouts.

• Learned basis matrices decompose each target state, while dedicated branches predict each component from shared context.
• Predictive regression preserves factor magnitudes so the components can be synthesized back into a complete latent state.
• Orthogonality, factor-activity, and online variance objectives discourage repeated directions, inactive factors, and encoder collapse.
• The framework covers temporal, spatial, and partial-observation prediction and is evaluated across vision, biology, healthcare, control, and molecular dynamics.