Twins: Learn to Predict Unified Representations with Focal Loss
Ranking
Overall
72
Content
85
Popularity
41
Observed public metrics from 1 member.
Merged summary
TL;DR - Twins unifies ViT semantic features and VAE synthesis latents in one continuous visual token space without increasing sequence length. A focal regression loss addresses optimization imbalance, improving ImageNet generation quality while retaining competitive multimodal understanding.
- Concatenates aligned ViT and VAE features channel-wise on the same token grid.
- Attributes training imbalance to frequency bias, intrinsic dimensionality, and differing uncertainty.
- Upweights high-error VAE dimensions during flow matching to balance representation learning.
- Improves ImageNet gFID by up to 10.57 over naive MSE without classifier-free guidance.
Sources (1)
Twins: Learn to Predict Unified Representations with Focal Loss
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - Twins unifies ViT semantic features and VAE synthesis latents in one continuous visual token space without increasing sequence length. A focal regression loss addresses optimization imbalance, improving ImageNet generation quality while retaining competitive multimodal understanding.
- Concatenates aligned ViT and VAE features channel-wise on the same token grid.
- Attributes training imbalance to frequency bias, intrinsic dimensionality, and differing uncertainty.
- Upweights high-error VAE dimensions during flow matching to balance representation learning.
- Improves ImageNet gFID by up to 10.57 over naive MSE without classifier-free guidance.