Transformers Struggle to Use Their Emergent World Models: Revisiting the Tower of Hanoi, and the Illusion of Thinking

TL;DR - An interpretability study of the Tower of Hanoi shows that both small in-house Transformers and frontier reasoning models build a linearly decodable, geometrically faithful world model of the puzzle (a Sierpinski triangle), but large models lose that representation during extended chain-of-thought — reframing the reported "illusion of thinking" collapse as a maintenance failure rather than an absence of world models.

• Small Transformers trained from scratch on precomputed solution traces develop an emergent, causally involved representation of the puzzle state space matching the Sierpinski triangle structure.
• Qwen3.6-27B and DeepSeek-R1-Distill-Qwen-32B encode this same world model near-perfectly at the end of the prompt, yet fail on the majority of flat-to-flat variants with more than 3 rings.
• Probing across planning stages localizes the failure to decaying world-model representations during generation; injecting the prompt-time representation at inference improves performance, establishing causality and partial recoverability.
• The flat-to-flat variant (arbitrary initial and goal configurations, not all rings on one peg) is the discriminating task where current LRMs still break down.