MODUS: Decoder-Only Any-to-Any Modeling of Diverse Modalities

TL;DR - MODUS is a decoder-only any-to-any model that handles arbitrary modalities symmetrically as inputs and outputs. It enables flexible cross-modal generation while leveraging pretrained decoder-only models.

• Uses one network without modality-specific heads, losses, or task pipelines.
• Supports chained generation through intermediate modalities and cross-modal self-verification.
• Achieves competitive performance against specialist and multitask baselines across multiple benchmarks.
• The authors have open-sourced all materials.