\k{appa}-LoRA: Condition Numbers Reveal Which LoRA Matrices Worth Updating
Ranking
Overall
68
Content
80
Popularity
41
Observed public metrics from 1 member.
Merged summary
TL;DR - κ-LoRA selectively updates LoRA matrices with the highest condition numbers, preserving standard LoRA accuracy while reducing fine-tuning costs.
- Updating only the top 50% of matrices halves the trainable parameter count.
- Experiments report 16.2% lower fine-tuning time and 4.5% lower memory usage on average.
- High-condition-number matrices contain underdeveloped directions that contribute most to adaptation.
- Their condition numbers decline during training, suggesting targeted spectral rebalancing drives the gains.
Sources (1)
\k{appa}-LoRA: Condition Numbers Reveal Which LoRA Matrices Worth Updating
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - κ-LoRA selectively updates LoRA matrices with the highest condition numbers, preserving standard LoRA accuracy while reducing fine-tuning costs.
- Updating only the top 50% of matrices halves the trainable parameter count.
- Experiments report 16.2% lower fine-tuning time and 4.5% lower memory usage on average.
- High-condition-number matrices contain underdeveloped directions that contribute most to adaptation.
- Their condition numbers decline during training, suggesting targeted spectral rebalancing drives the gains.