Grounding latent algorithm routing in transformer reasoning
Ranking
Overall
84
Content
90
Popularity
70
Observed public metrics from 1 member.
Merged summary
TL;DR - ROUTEBENCH shows that dense transformers trained from scratch can learn internal routing behavior that selects among solver families based on the latent data regime. This provides controlled evidence for algorithm-like adaptation during in-context learning, without claiming the behavior generalizes to pretrained LLMs.
- A 306M-parameter model closed 80.9% of the oracle-routing gap and achieved 84.1 route F1.
- Routing differentiated ridge-, lasso-, Huber-, and kNN-like strategies associated with shrinkage, sparsity, robustness, and locality.
- The behavior persisted across natural-language renderings, shuffled examples, lexical paraphrases, and unified four-way routing.
- Probing and activation patching indicated that route-related internal directions were both decodable and functionally involved in outputs.
Sources (1)
Grounding latent algorithm routing in transformer reasoning
Public signals
Semantic Scholar citations 1 · Semantic Scholar influential citations 0
TL;DR - ROUTEBENCH shows that dense transformers trained from scratch can learn internal routing behavior that selects among solver families based on the latent data regime. This provides controlled evidence for algorithm-like adaptation during in-context learning, without claiming the behavior generalizes to pretrained LLMs.
- A 306M-parameter model closed 80.9% of the oracle-routing gap and achieved 84.1 route F1.
- Routing differentiated ridge-, lasso-, Huber-, and kNN-like strategies associated with shrinkage, sparsity, robustness, and locality.
- The behavior persisted across natural-language renderings, shuffled examples, lexical paraphrases, and unified four-way routing.
- Probing and activation patching indicated that route-related internal directions were both decodable and functionally involved in outputs.