「有些模型就是不想学？」循环模型为什么越想越错？

TL;DR - This article surveys recurrent language models that repeatedly refine hidden states to gain test-time compute without generating longer chains of thought. Extra recurrence can improve reasoning, but exceeding trained depths often destabilizes representations and degrades accuracy.

• Latent-reasoning models reportedly scaled to 3.5B parameters and benefited from additional recurrent computation on GSM8K and HumanEval before saturating.
• Ouro-Thinking 1.4B peaked at 65 on AIME 2024 after four loops, then fell to 38.67 after eight.
• Repeated hidden-state updates can cause accumulated errors, exploding residual norms, loss spikes, and gradient oscillations.
• Parcae’s stabilization mechanisms reduced validation perplexity by up to 6.3% and beat standard Transformer baselines at 1.3B parameters under matched parameter and data budgets.