DeepMind又改Transformer了：深层激活回流，小模块反超全量微调

TL;DR - DeepMind’s Recirculation method feeds deep-layer Transformer activations back into shallower layers so later tokens can reuse more mature contextual states. A small adaptive controller improved Gemma3 language modeling beyond full fine-tuning while keeping the base model frozen.

• Adaptive Recirculation reduced average perplexity by 23.0% across nine language-modeling datasets, versus 21.6% for full fine-tuning and 8.5% for fixed recirculation.
• The method mixes norm-aligned deep- and shallow-layer states; a token-conditioned MLP dynamically predicts per-dimension mixing coefficients.
• Improvements appeared across several Transformer families, but were strongest on Gemma3 and less consistent on downstream tasks.
• Generation latency was reportedly negligible because computations can run in parallel, while sequential state updates make prefill slower for long contexts.