Learning on the Job: Continual Learning from Deployment Feedback for Frozen-Weights Agents

TL;DR - Frozen-weight agents can continually improve from deployment feedback by distilling completed episodes into retrievable natural-language rules. On τ-bench banking tasks, this external memory substantially outperformed static RAG without modifying model weights.

• One-bit outcome verdicts raised single-trial success to 1.6× the static-RAG baseline; corrections raised it to 2.6×.
• Correction-based learning solved 22 of 84 tasks that the baseline never completed.
• Results held for self-hostable Mistral Large and frontier-model Claude Sonnet 5.
• Memories transferred across models, improving each model over its own no-memory baseline.