dQwen3.5: Hybrid-Attention Diffusion Language Models

TL;DR - dQwen3.5 adapts Qwen3.5’s hybrid attention–RNN architecture into diffusion language models ranging from 0.8B to 9B parameters. The results suggest hybrid autoregressive backbones can enable efficient diffusion-model adaptation despite their structurally causal RNN layers.

• The family spans 0.8B, 2B, 4B, and 9B parameter scales.
• Hybrid models reach a given training loss using roughly half the tokens required by a full-attention control.
• Across scales, dQwen3.5 exhibits any-order decoding behavior similar to full-attention diffusion language models.
• The adapted models perform strongly with parallel decoding.