Does Your Agent's Memory Survive a Model Upgrade? A Controlled Study of Memory Portability
Ranking
Overall
81
Content
100
Popularity
37
Observed public metrics from 1 member.
Merged summary
TL;DR - This controlled study finds that an AI agent’s stored memory may not transfer reliably when its underlying model changes. Fixed-schema knowledge graphs were highly portable, while model-generated notes and partially migrated RAG indexes suffered substantial degradation.
- KG-fixed accuracy changed by just (+0.0004 \pm 0.0020) after a writer-model swap.
- NOTES were strongly model-coupled, with migration changing accuracy by (+9.91) or (-13.28) percentage points depending on direction; 80% of their deficit came from information lost during construction.
- A 50/50 mixed-embedding RAG index gained only 4.96 points versus 11.90 points from full re-embedding, with retrieval failures accounting for 81% of the RAG deficit.
- Store-only NOTES repair never achieved 90% recovery across 48 histories, while retaining raw histories enabled recovery in 34 cases for one migration direction.
Sources (1)
Does Your Agent's Memory Survive a Model Upgrade? A Controlled Study of Memory Portability
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - This controlled study finds that an AI agent’s stored memory may not transfer reliably when its underlying model changes. Fixed-schema knowledge graphs were highly portable, while model-generated notes and partially migrated RAG indexes suffered substantial degradation.
- KG-fixed accuracy changed by just (+0.0004 \pm 0.0020) after a writer-model swap.
- NOTES were strongly model-coupled, with migration changing accuracy by (+9.91) or (-13.28) percentage points depending on direction; 80% of their deficit came from information lost during construction.
- A 50/50 mixed-embedding RAG index gained only 4.96 points versus 11.90 points from full re-embedding, with retrieval failures accounting for 81% of the RAG deficit.
- Store-only NOTES repair never achieved 90% recovery across 48 histories, while retaining raw histories enabled recovery in 34 cases for one migration direction.