T^2MLR: Transformer with Temporal Middle-Layer Recurrence

TL;DR - T²MLR is a Transformer architecture that carries a cached middle-layer representation from the previous token into an earlier layer of the current token, letting abstract reasoning states persist across decoding steps with little inference overhead. It matters because it improves latent reasoning without full-layer looping or pretraining from scratch.

• Fuses a previous token's cached middle-layer representation into an earlier layer of the current position, enabling intermediate computation to persist across autoregressive steps.
• Outperforms data- and parameter-matched Transformer baselines on natural-language pretraining and multi-hop reasoning finetuning.
• Localized recurrence over ~20% of the network (a middle-layer block) often beats full-layer recurrence.
• Can be retrofitted into an existing pretrained 1.7B Transformer with brief finetuning, substantially improving math reasoning and lowering adoption cost.