\k{appa}-LoRA: Condition Numbers Reveal Which LoRA Matrices Worth Updating

TL;DR - κ-LoRA selectively updates LoRA matrices with the highest condition numbers, preserving standard LoRA accuracy while reducing fine-tuning costs.

• Updating only the top 50% of matrices halves the trainable parameter count.
• Experiments report 16.2% lower fine-tuning time and 4.5% lower memory usage on average.
• High-condition-number matrices contain underdeveloped directions that contribute most to adaptation.
• Their condition numbers decline during training, suggesting targeted spectral rebalancing drives the gains.