「有些模型就是不想学？」循环模型为什么越想越错？
Ranking
Overall
77
Content
80
Popularity
70
Observed public metrics from 1 member.
Merged summary
TL;DR - This article surveys recurrent language models that repeatedly refine hidden states to gain test-time compute without generating longer chains of thought. Extra recurrence can improve reasoning, but exceeding trained depths often destabilizes representations and degrades accuracy.
- Latent-reasoning models reportedly scaled to 3.5B parameters and benefited from additional recurrent computation on GSM8K and HumanEval before saturating.
- Ouro-Thinking 1.4B peaked at 65 on AIME 2024 after four loops, then fell to 38.67 after eight.
- Repeated hidden-state updates can cause accumulated errors, exploding residual norms, loss spikes, and gradient oscillations.
- Parcae’s stabilization mechanisms reduced validation perplexity by up to 6.3% and beat standard Transformer baselines at 1.3B parameters under matched parameter and data budgets.
Sources (1)
「有些模型就是不想学？」循环模型为什么越想越错？
Public signals
Hugging Face upvotes 162
TL;DR - This article surveys recurrent language models that repeatedly refine hidden states to gain test-time compute without generating longer chains of thought. Extra recurrence can improve reasoning, but exceeding trained depths often destabilizes representations and degrades accuracy.
- Latent-reasoning models reportedly scaled to 3.5B parameters and benefited from additional recurrent computation on GSM8K and HumanEval before saturating.
- Ouro-Thinking 1.4B peaked at 65 on AIME 2024 after four loops, then fell to 38.67 after eight.
- Repeated hidden-state updates can cause accumulated errors, exploding residual norms, loss spikes, and gradient oscillations.
- Parcae’s stabilization mechanisms reduced validation perplexity by up to 6.3% and beat standard Transformer baselines at 1.3B parameters under matched parameter and data budgets.