Selective State-Space Adaptation and Retrieval for Language Model Reasoning

TL;DR - MaLoRA and MaRA add selective state-space recurrence to frozen language-model adapters, enabling input-dependent token adaptation and retrieval across context segments. They consistently outperform standard LoRA on multi-hop reasoning benchmarks.

• MaLoRA dynamically modulates adapter scaling using recurrent token-level state.
• MaRA maintains cross-segment state and selects query-relevant context before generation.
• Across three 7B–9B backbones and two benchmarks, the methods improve average F1 by 6.8 points over LoRA.
• Gains reach 9.3 F1 on the hardest setting and transfer to long-context RULER QA-2 tests.