UEmbed: Unified Sparse and Dense Multimodal Embeddings

TL;DR - UEmbed is a decoder-only multimodal embedding model that emits both learned-sparse lexical and dense representations in a single causal forward pass, removing LSR's dependence on bidirectional encoders and auxiliary cross-modal modules. It matters because it collapses two retrieval paradigms (sparse + dense) and two modalities (text + image) into one unified model.

• Appends N learnable special tokens and partitions the vocabulary into N disjoint subsets; each token's causal hidden state predicts sparse weights over its subset, and the subsets concatenate into the full sparse vector.
• Released at 2B, 4B, and 9B scales, trained only on public data.
• UEmbed-9B scores 71.8 (dense) and 71.0 (sparse) on MMEB-v2, outperforming public-data multimodal embedding models such as RzenEmbed; stays competitive with strong dense and sparse baselines on BEIR.
• Authors evaluate utility along effectiveness, efficiency, and agentic applications, positioning it for RAG and search pipelines.