下一场AI革命，要取代Transformer

TL;DR - AI startups are developing alternatives to dense Transformer architectures—such as sparse attention, retention, liquid neural networks, diffusion, and state-space models—to improve efficiency and reasoning. These approaches matter because current Transformer gains increasingly rely on costly engineering workarounds.

• Subquadratic claims its sparse-attention mechanism competes with leading models on some search and coding tasks.
• Liquid AI’s hybrid LFM architecture uses roughly 20% Transformer and 80% liquid neural networks, enabling deployment on constrained hardware.
• Pathway’s Dragon Hatchling replaces attention with state-space representations designed to support abstract, non-token-sequential reasoning.
• The reported alternatives remain emerging approaches rather than proven general replacements for Transformers.