Past, Future, All at Once: Mitigating Stability-Plasticity Dilemma via Post-hoc JANUS Rectification
Ranking
Overall
82
Content
95
Popularity
N/A
No observed public metrics; popularity remains neutral/archived.
Merged summary
TL;DR - JANUS is a post-hoc, fine-tuning-agnostic method that projects parameter updates into a model’s Jacobian null space to recover historical knowledge after adaptation. It addresses catastrophic forgetting while aiming to preserve performance on newly learned tasks.
- Establishes Parameter Space Orthogonality as the necessary and sufficient first-order condition for preserving historical performance.
- Uses multi-step adaptive rectification to verify the Jacobian approximation’s trust region and dynamically adjust step sizes.
- Introduces ghost projection, orientation comparison, and sequence-level SVD compression to improve temporal and spatial efficiency.
- Experiments show compatibility with multiple fine-tuning methods and reduced forgetting without disrupting downstream adaptation.
Sources (1)
Past, Future, All at Once: Mitigating Stability-Plasticity Dilemma via Post-hoc JANUS Rectification
Public signals
N/A
TL;DR - JANUS is a post-hoc, fine-tuning-agnostic method that projects parameter updates into a model’s Jacobian null space to recover historical knowledge after adaptation. It addresses catastrophic forgetting while aiming to preserve performance on newly learned tasks.
- Establishes Parameter Space Orthogonality as the necessary and sufficient first-order condition for preserving historical performance.
- Uses multi-step adaptive rectification to verify the Jacobian approximation’s trust region and dynamically adjust step sizes.
- Introduces ghost projection, orientation comparison, and sequence-level SVD compression to improve temporal and spatial efficiency.
- Experiments show compatibility with multiple fine-tuning methods and reduced forgetting without disrupting downstream adaptation.