Mamba with Hierarchical Memory: Solving Representation Bottleneck in Long Sequence Modeling

TL;DR - An arXiv preprint proposing Hierarchical Memory Mamba (HMM), which bolts a brain-inspired working/long-term memory hierarchy onto a pre-trained Mamba backbone to fix the fixed-capacity recurrent state bottleneck that hurts linear-attention models on long sequences.

• Treats the Mamba backbone's hidden states as fast "sensory memory," adds a lightweight working memory that extracts slow paragraph-level semantics (PLS), then compresses PLS into a persistent long-term memory for task-relevant retrieval.
• Reported gains over strong Mamba-based baselines: +34.3–37.1% retrieval success on Passkey Retrieval and +1.6–14.2% reasoning accuracy on LongBench-E.
• Costs only ~2% extra parameters with minimal training overhead, since it builds on an existing pre-trained backbone rather than training from scratch.
• Authors claim cross-task generalization via parametric learning, which they say is absent in other long-context Mamba variants.