Grounding latent algorithm routing in transformer reasoning

TL;DR - ROUTEBENCH shows that dense transformers trained from scratch can learn internal routing behavior that selects among solver families based on the latent data regime. This provides controlled evidence for algorithm-like adaptation during in-context learning, without claiming the behavior generalizes to pretrained LLMs.

• A 306M-parameter model closed 80.9% of the oracle-routing gap and achieved 84.1 route F1.
• Routing differentiated ridge-, lasso-, Huber-, and kNN-like strategies associated with shrinkage, sparsity, robustness, and locality.
• The behavior persisted across natural-language renderings, shuffled examples, lexical paraphrases, and unified four-way routing.
• Probing and activation patching indicated that route-related internal directions were both decodable and functionally involved in outputs.