Twins: Learn to Predict Unified Representations with Focal Loss

TL;DR - Twins unifies ViT semantic features and VAE synthesis latents in one continuous visual token space without increasing sequence length. A focal regression loss addresses optimization imbalance, improving ImageNet generation quality while retaining competitive multimodal understanding.

• Concatenates aligned ViT and VAE features channel-wise on the same token grid.
• Attributes training imbalance to frequency bias, intrinsic dimensionality, and differing uncertainty.
• Upweights high-error VAE dimensions during flow matching to balance representation learning.
• Improves ImageNet gFID by up to 10.57 over naive MSE without classifier-free guidance.