MACRO: Markov Chain Routing of Transformer Layers

TL;DR - MACRO is a training-free framework that learns task-specific execution routes through a frozen LLM's transformer layers, modeling routing as a context-dependent Markov policy. It matters because it extracts meaningful accuracy gains without touching model weights or running per-instance search.

• Routes are conditioned on layer index, computation-budget phase, directional displacement, and operator context, supporting skip, repeat, and residual hidden-state addition moves.
• The Markov route distribution is updated from training-data feedback and decoded with a top-k Viterbi algorithm to extract high-probability candidate "programs."
• Reported results: +5.0% average accuracy over unrouted baselines across reasoning and knowledge benchmarks on multiple open-weight LLMs, with the largest gains on small models.
• Beats prior dynamic-routing method Dr. LLM by +7.2% while cutting route-search time 9.4x (14.8 → 1.6 hours); code released at github.com/Batorskq/MACRO.