LiveMem: Maintaining Memory State Continuity in Long-Running LLM Inference

TL;DR - LiveMem is an arXiv preprint proposing an "intrinsic memory" mechanism that gives a pretrained full-attention LLM a fixed-capacity memory state whose lifetime outlives the active KV context, so long-running assistants/agents can keep computing over history that has already scrolled out of the window. It matters because it reframes long-context handling from retrieval/summarization of past text to maintaining persistent inference state.

• Formulates the problem as state continuity under context turnover: carrying computation forward in a fixed-capacity memory state whose lifetime is decoupled from the active context, unlike retention, summarization, or retrieval which only preserve access to selected history.
• Architecture keeps the main attention path on a bounded KV window while a persistent memory state absorbs historical information across the full lifecycle.
• Three components jointly make the state load-bearing after its originating tokens are evicted: context turnover / memory-state maintenance, memory-oriented post-training, and state-aware serving.
• Reported results: leading overall performance versus evaluated systems and other intrinsic-memory methods; on LongMemEval it answers questions after supporting evidence is removed from context, and evidence-distance analysis shows information persisting beyond the active window (no specific metric values given in the abstract).